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General Summary 

Introduction 

Project RFD is a demonstration project in Adult Basic Education 
funded under the provisions of Section 309(b) of the Adult Education 
Act of 1966. The project has operated during the 1970, 1971, and 
1972 fiscal years with a total funding of $708,000. The project has 
operated at the University of Wisconsin - Extension which is in Madison, 
Wisconsin. 

The external evaluation was conducted by a team of five persons 
from Colorado State University, Fort Coljiii^ Colorado. The evalu- 
ation work was funded by project monies with a sub-contract written , 
between Colorado State University and the University of Wisconsin. 
The contract was made on the basis of a proposal submitted by the 
team to a Request for Proposal for evaluation issued by Project RFD. 
This report presents the findings and judgments of the team. 

The following persons were the team members: all from Colorado 
State 'University. 

Mr. Preston Davis, Director, t)ffice of Educational Media 
Dr. Mary "Helen Haas, Professor of Vocational Education 
Dr. Ronald Wykstra, Professor pf Economics 
Dr. Henry Cross, Professor of Psychol Ofly and co-director of 

the evaluation study. 
Dr. Douglas Sjogren, Professor of Education and co-director 

of the evaluation study. 

A considerable amount of description of the project is included 
in this report. It is recommended, though, that the person who wants 
in-depth descriptive information obtain copies of the reports listed 
in the reference list at the end of this report. 



Purpose and Method of the Evaluation Study 

Our perception of the purpose of the External Evaluation study was 
that of validation. The primary concern seemed to be one of having 
an independent group of knowledgeable persons observe, study, and make 
judgments about the project with respect to its internal operation 
and its potential application. Such judgments have been made by ^he 
Project RFD staff on the basis of information gathered in a quite 
comprehensive internal evaluation effort. The question for the externa 
evaluation then is whether an independent group will arrive at the same 
judgments using essentially the same information base. 
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The overall purpose of the external evaluation is reflected in the 
six purposes stated in the ca^l for sealed bids issued by the project. • 
The purposes-, listed below, served as the objectives of the external 
evaluation study. 

1. To review the accompl istiments. of the project in' design, field 
testing, and evaluation. _ 

2. To assess ^.ho extent to which the project achieved its objetti\^es. 

3. To assess he extent to which data collection has been adequate 
to the requirements of the research. 

4. To assess and make judgments about the design and execution 
of the internal research. 

5. To relate cost to accomplishments. . 

6. To prepare a report that will be readily incorporated into the 
final project report. <j 

The team operated much like an accrediting team. Much of the 
information for the evaluation study was in the form of reports 
issued by the Project RFD staff. They have done a good job of pre- 
paring descriptive narrative reports of the project as it has proceeded. 
The narrative not only describes what occurred, but also includes 
the rationale for the decisions that were made. 

The Psychometric Laboratory at the University of Wisconsin did 
the internal evaluation for the project. The lab has produced several 
reports on various 'aspects of the project. These reports were avail- 
able to the team, and provided valuable information for the external 
evaluation. 

- Members of the team visited the project and interviewed many 

people who were associated with the project. A total of 10 man-days 
were spent on the site by the team and about 50 persons were inter- 
viewed Those interviewed included project staff, former staff, 
participants, ABE personnel, advisory committee members, and several 
persons in the University hierar*chy. The information obtained 
in the interviews was an important supplement to that available in 
the reports. 

During the time of the study, the team met several times to review 
the information that had been obtained, and to decide on information 
needs These meetings permitted the team members to check their impres- 
sions 'against the others, and to stimulate the thinking of the team. ' 
Generally, the team meetings occurred before and after a member or 
members visited the project. 

From a methodological point-of-view we f^lt that having all 

the evaluation team members from one institution facilitated the study. 

We were able to interact quite often over a pferiod of time. This seemed 

to help us to clarify our thinking and check our perceptions of the 
project. 
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section of the report contains a' brief description of the project 
and a summary sta^tement on^the .-attainment of the project goals. The 
subsequent sections are ^essentially critiques of the conceptualiza- 
tion, development and o^'peration of each of the components of the project. 
The final section is a QQst analysis of the project. 



. P^iect RFD was conceived as a method or model for delivering ^ 
educational opportunity to the rural adult. More 'specif ically the / 
project proposed to deliver educational opportunity to the rural / 
adult who is undereducated, i.e. Adult Basic Edication. Essential li^ 
the project was a demonstration and test of two related ideas. First, 
it was a demonstration and test of a delivery system that might 
have many kinds of application. Second, it was a demonstration and 
test of materials designed to provide educative experiences for 
undereducated adults. 

The needs for' the delivery system and for the educative experiences 
were auite w^ell documented in the first proposal. Furthermore, the 
rationale for the model and for the experiences seemed to be sound. 
The proposed demonstration and test was well -conceptualized and 
seemed to have'a reasonable probability of success even though it should 
be considered a high-risk type of project. The models are detailed in 
references one and two. 

The fidelity of the project as it was conducted with its conceptual- 
ization was fair. Several changes were made in both of the basic 
ideas from conceptualization to operation. The basic ideas, 
however, did persist and the project as conceptualized was certainly 
recognizable in the operation. 

The critiques of each component present specific detail on the 
changes from conceptualization to operation. A summary of the 
changes is presented here for those readers who only want an over- 
view. 

Quoting from the first-year proposal the original goal of the 
project was: ' 

"Development of a... rural adult basic education and continuing 
education demonstration and research project utilizing educa- 
tional television, individualized home study instruction tech- 
niques, and a personalized home contact instruction and evalu- 
. ation plan 

...By combining these three elements, the University and State 
of Wisconsin propose to demonstrate a new approach to providing 
adult basic education courses for the rural disadvantaged," 



Overview 
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The television component was regarded as the "basic instructional * 
and motivational tool". . The half-hour programs were'to be designed* "to 
introduce educational concepts of importance to the entire student 
group and to motivate the student prior to the programmed, variable- 
speed, home study phase of the instruction." 

The home study component was intended to consist of materials ""^^X 
that would be used by the S'tudent in his home setting much in the *^ 
nature^ of correspondence courses. •They were to be designed so 
that the student could proceed at his own pace. Several areas of study 
were intended including ba|ic literacy, family life and consumer 
education, social relations, conservation, enjoyment of life, an^ 
others. The materials were to be for those with elementary level read- . 
ing skills. 

The home-contact component was intended as a procedure for 
instructing, motivating and^counseling, and for evaluation of the 
student's progress. The "Home Study Aide" was to provide direct 
instruction. 

In the ideal sense, the intent was that a student would view a 
TV program that would provide some basic instruction and stimulate 
interest. Then the student would receive the home-study materials 
that were appropriate.' The home visitor would interact with tbe 
student weekly and answer questions, instruct, counsel, and provide 
incentive for persisting in the program. 

'-The first year of the project was devoted to the development 
of the materials and procedures , for operational izing the demonstration. 
Several decisions were made during the first year that resulted in 
changes in the conceptual design. The three main components; tele- 
vision, home-study, and home visit were still included and some new 
components were added; an action line, a newspaper or almanac, and 
a radio show. 

The intent for the television component changed somewhat. Surveys 
were conducted on viewing likes and dislikes. On the basis of the 
surveys and caireful consideration of the potential of television, it 
was decided that the TV component would be designed to serve pri- 
marily in a motivational function. A "magazine show" format was 
developed for the twenty half-hour programs that were used in the 
demonstration. The shows contained interviews'^ith personalities, help- 
ful hints on buying, cooking, home repair, etc.,*and short spots on 
topics like conservation, insurance, social security, etc. . 

The home study component changed considerably during the first 
year. On the base's of surveys and consul tation .two major decisions 
were made. First, the emphasis changed, "Whereas literacy skills 
were emphasized in the original proposal, the intent of the home 
study materials after the first year of work- had changed to a 
broader concept of literacy. The primary emphasis shifted to materials 
on what were called coping skills. 
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Coping skills were defined as the skills and knowledge needed 
to interact effectively in one's environment,' It was decided that 
materials v^o^^4 be made a^-allable in five general areas called 
content center?. The content center topic areas were entitled, • 
''About Me", "About Me and Others", "About Me and My Money", 
"About My Work", and "Me and My Community. Materials on the 
usual literapy skills, the three R's, were to be available still, 
but only on specific requestor on a decision made by the student 
with the home visitor. 

The second major decision was with respect to assumptions about 
^e reading level of the coping. skills materials. The original plan 
was to develop materials for reading levels 0-8. Actually, it 
was intended that the materials be written at three levels roughly 
equivalent to K-2, 3-5, and 6-8, On consultation, it was decided 
to preparedly one set of reading materials with a reading 
equivalency of about grade 5, -This decision was probably a good 
one from the practical viewpoint in that it reduced the amount of ^ 
materials that had to be produced. On the other hand, the decision 
did compronfise the original Adult Basic Education objective somewhat. 

The conieptual'ization of the home visit component also changed 
during the first year ..planning period.* The indent that the home 
visitor would be a motivator, counselor, and friend was maintained 
but the instructional function of the home visitor was de-emphasized. 
One fact about the home visit component became obvious during the 
planning year. This was that there was a very real limit on the 
number of 1iome visits that could* be made: It was decided that the 
home visit component would be used with 50 persons in the demonstration. 

With respect to the action line, almanac, and radio show., it is 
not clear from the reports just when these were conceptualized. The 
action line component is ^described in the second year report and is. 
rationalized as a feedback linkaqe ;:etween the participant and the 
program. The learner audience had a direct communicative link with ^ 
RFD and the members were provided a means for solution of personal 
problems. 

The purpose of the almanac is not clearly enunciated in any report. 
It appears to have been designed as a motivational device in that 
it is a procedure for regular contact with the audience. The speci- 
fic need for which it was designed is not clear to us, however. 

The radio program is not described either. Apparently its 
purpose was to do some of the same things as the ^TV programs in 
homes where there was no TV. We feel this medium might have been 
used more than it was . 



The second year of . the project was the development^ and -demon- 
stration year. The twenty TV tapes were produced, the home study 
materials were selected and/or written,. the home visitors were 
'hired and trained, the evaluation procedures were developed, and 
the sample for the demonstration study was selected. All was 
upposedly in readiness for the demonstration study by January 1. : 
here was some slippage, however, and although the demonstration 
study started on schedule, it was hampered by. the fact that all was 
not ready. 

Actually there were two demonstration studies. One study was an 
experiment in which two random saqiples with an N of 50 in each were 
drawn from a population of an identified target audience in the four 
counties served by WHA-TV. One sample, hereafter called the treatment 
group, had acces?- to all components. The received the TV programs, . 
received the home study materials, had home visitation, ^and had 
access to the action line, almanac, and radio program. The other 
sample, the control group had access to every component except the 
home visitation. The sampling procedures were such that this 
study appeared to be a. well-controlled experiment in which the 
independent variable categories were having or not having home 
visitation. The critique of the design section of this report indi- 
cates some. of the problems that were encountered in this ^study. 

The second demons tratiorv study was done with th'e rest of the popu- 
lation in the WHA-TV viewing area. The entire population had access 
to every component but the home visitation. Extent of participation 
was studied. This study is also critiqued in the design section 
of this report. 

The third year of the project has been devoted to analyzing and 
reporting the evaluative information, revising ancJ writing materials 
for national dissemination, conducting dissemination .activities, and 
planning nextst^ps." ^ 

Was Project RFD successful? It has been said in many^ways that 
there are many si.des to truth. So it is with RFD. It was not a 
failure nor was it the success some might have hoped. Some things 
were done very well, some things were not done at all as expected. 
A model was implemented and educational experiences were delivered, but 
the model is not, nor should it be expected to be, the final solution 
to adult education. 

A reading of the critique sections will reveal the specific 
reasons for our middle-of-the-road conclusions. As a conclusion 
to this summary statement, however, we have reviewed each of the 
original objectives of the project and offered our judgment of the 
extent to which the objective was attained. 
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Objective One - Demonstrate the effectiveness of an integ rated tele- • 
vision, home study^ home conftact and visit program for rural ABE students . 

This objective was. really the overall goal of the project. 
"This objective was attained to some extent in that the project did 
develop and carry through a method for delivering educational activities I 
to adults. Furthermore, the delivery system did include all the com-, 
portents . 

Three key words in this objective are "effectiveness", "inte- ^ 
grated" and "rural". An attempt was made to integrate the components, 
but the integration was not effected well in the demonstration. There 
was little evidence that the TV programs wece designed to stimulate 
interest in materials in the content center.' The home visitors and 
the participants with whom we visited saw little or no correlation 
among the three components except for the home visitor helping the 
participant obtain materials. "The TV programs did include spots 
where materials in the content center were mentioned and information 
was provided on how to obtain the materials. New materials were 
described each week, however, and it is unlikely that the participant 
in. the target audience would be ready for new material on a weekly 
basis. *A"lso any single JV program , did not reveal .all of the materials 
that were available ^so the participant; was not really aware of all 
' of the alternatives^ Of course," those participants who had the . . 
home visitor were likely made «ware Of. the alternatives by the home 
^ v,isit;ors. The TV programs were very much designed with.. a "soft-sell" 
approach, perhaps too soft. 

The project did many things to stimulate contact between the 
potential participant and the project, and .these activities .did serve 
an integrative function. The action line, post card contacts, and 
the extensive publicity in the project did serve to increase awareness 
of the many things that Project RFD had to offer. 

Our judgment -with respect to the integration attempt is. that 
the project staff made a concerted effort to achieve integration of 
the components, but were not as successful with this intent as might 
have been desired. They did learn much about the problem of inte- 
grating multi -media approaches. . • 

* 

The second key word is effectiveness. This word implies some 
criteria which are not specified in this objective., The criteria 
are suggested, however, by some of the -other objectives, and the 
discussion of those objectives suggests the extent of effectiveness of 
ttie project. In summary, the project did have a high level of 
participation in the four county area. There is some question, however, 
whether the participation rate was high in the target audience. 
Many people requested materials. The? requests for materials were 
significantly higher in the group that had the home visitor than 
in the control group or the rest of the population. The home visit 
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component ap'parently,was effect'i've in getting educational materials 
into the hands of the participant. There was litt e evidence, however, 
that there was any change tn skill, knowledge, or attitude due to 
^he project or any of its components. 

The last key word, is /ural.' Jhe project was originally designed 
for a rura"" audience. This intertt was not maintained, however, and 
the project was really for the total population. In fact., it 
appeared that in terms of^ absolute numbers and also proportionately, 
the participants genera.l.ly Kere frOni the metropolitan area of 
Madison. Furthermorp,', the .-100 subjects in.^the experiment were pri- . 
marily from town|,in;the area rather than from farms. This does not . 
ind-icate that^ the delivery system and materials would be ineffective 
with a rural audience. We still don't know. ^ 

On a- 5 point scale of attainment, with 5 indicating a high degree 
of attainment, we rate the 'attainment of this objective 3.0 to 3.5. 

Objective Two - Demonstrate the effectiveness of the r^le of mas s 
media in rural ABE programs. 

This objective is not clear to us, again because of the terms 

"effectiveness", "rdle", and "rural". We have addressed the problems 

with the terms effectiveness and rural in the discussion of the first 

objective. The term "role" is problematic because there is no 
definition of just what the role is conceived to be. 

If the^rble of mass media is to stimulate 'and motivate, then 
there is some evidence that, this occurred. Many people watched 
the programs. A large majority of those who watched liked the 
programs. Many of the watchers, were stimulated to contact the 
project for materials-. These observations do indicate that 
assigning a stimulation role to mass media is effective. 

With the definition of the role used here and disregarding the 
rural audience intenti;j-V/e rate the attainment of this objective at 3.5 to 
4.0. . ' ^ 

Objective Three - Create a viable television-based multi-media program 
usable in similar situations in other parts of the country . 

Tne project did create and demonstrate a television^based multi- 
media program. They have also developed a plan for use of the model 
in other areas along with co§t estimates. The rationality or feasi- 
bility of the model is attested to by the fact that several states and 
other agencies have indicated an interest in using cfll or parts of the 
' model. The project staff certainly believes the. mod6l is viab>e. 
They have developed a proposal to implement the model in a^ca'reer 
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education program. Many persons whom we interviewed in the University 
a-^l who were no^ directly associated with the project, indicated a 
feeling that the model seemed to have promise as a method for the 
outreach function of the University. 

The viability of the program in other situations is very much 
related to costs. The cost of the television and materials componentr 
would be reasonable if they are used by a large number of people 
over a period of time. The cost analysis of the project showed the 
fixed costs of these two components to be quite high but if the 
costs are .anortized ovor a number of people and over time they are 
^-^ relatively low per unit. The same is tru& of the almanac and the \^ 
radio show. The fixed costs for the home visit component are quite \ 
high as are the variable costs. This component is expensive and its 
cost increases proporticnately to time and persons served. The action 
line component was relatively inexpensive in the project because of 
volunteer help. If the labor costs for action line were to have been 
real it likely would have been quite costly. 

The geu€^alizability of the materials is another issue with this 
objective. 3^e TV programs produced flfif RFD are not general izable. 
They contain specific references to events and places in Wisconsin. >p 
On the other hand, there are parts of each tape that might be used 
in other situa'tions. The staff of the project has studied the issue 
of general izability. Their position now seems to'be that spots or 
short sequences might be developed for general distribution, but 
that the total program should be put together so that it uses local 
talent and "deferences. It sK uld be^evident that ^ the 
generalizability issue is related to costs. It also may be related to 
effects. A general tape would be usable in many situations and over > 
a period of time so that its cost could be amortized to a relatively 
low cost per unij. Specific tapes would have to be done over 
so that production costs would increase as well as untt amortization 
costs. On the other hand, specific tapes might be more effective 
than general tapes in attracting and holding audiences. There is 
likely an optimum balance but what that point might be is not known. 

The home study materials seem to be generaVizable to many 
situat'ons. They do not contain specific references, and the topics 
are quite general. A publishing house has contracted to publish 
much of the material for generalizability of the home study materials. 

Our attainment rating of objective three is 3.5 to 4.0. 

Objective Four - Demonstrate the e f fectiveness of an interdisciplinary 
family and community oriented approach to rural ABE pripgrams . 

The effectiveness issue rears its ubiquitous head again. The 
materials were interdisciplinary in that they were on topics that were 
quite general and involved principles from several bodies of knowledge. 
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The materials • 2re also family and community oriented in that they 
dealt with concerns of f^-'-yday life: work, the family, the home, 
money, etc. The projc ' ieirionstrate effectiveness in this 
area in the sense that lals could be developed with the family and 
community orientation. Furthermore, the materials seem to be- 
rational, accurate, informative, and interesting. Unfortunately 
there is little evidence on their effectiveness with respect to 
participant behavior. 

Our attainment rating for this objective is 3.0 to 3.5. 

Objective Five - Involve large numbers of undereducated adults not now 
able or willing to participate in ABE programs. 

It is difficult to determine just how many undereducated 
adults did participate in the program. From one point-of-view it 
could be argued. that every adult who contacted RFD was undereducated 
in some degree. They had a need or want for information ^nd ini- 
t^ "ed action to obtain it. Some 3,000 adults did contact RFD for 
so... thing during the five month demonstration. This is a large number. 

Undereducated usually connotes a person with little education, 
however. Excluding the 100 persons in the experiment, the 
available figures suggest that about 15% of the contractors were 
individuals with less than an eighth grade education, mis 15% 
figure is very near to the percent of persons with less than an 
eighth grade education in the four county area. The program did 
not attract a dis^proportionately large number with little education, 
but it also did riot have a disproportionately small representation 
from this educational category. The typical situation is that 
educational programs will attract persons with education and. those 
with little education are notable in their nonparticipation: 
Project RFD seems to have broken tl.is pattern somewhat. We 
would predict that if the demonstration had operated for longer 
than five months, the participation rates of the typical undereducated 
, adults would have increased. 

The experimenta'1 study had some interesting results in terms of 
involvement. The 50 persons in the treatment group persisted .wel 1 
in the project. There were 12 drop-outs but 8 of these were in the 
last month. The reasons>for drop-outs were generally like moving, 
working full-time, or family problems. There w«jc£ two refusals. 
Th.e treatment group had a high rate of participation in terms of 
viewing the programs and using the materials. Those m the control 
groupHjad a higher rate of participation than the general population 
but notSiS high as the treatment group. Two conclusions are suggested: 

1 The special attention given the control group during the recruit- 
ment phase seemed to stimulate them to participate even though 
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they did not have the home contact. This result suggests that 
some form of a stated commitment is conducive to participation 
and persistence. For example, enrollment procedures are a form 
of a commitment that might have a beneficial effect. 

2. The home contact component was effective in stimulating 
involvement and participation. Exposure to education is 
an important first condition. In the five month demonstration, 
the home contact component had a positive impact on exposure. 
Furthermore^, the efforts expended in publicity and adver- 
tising by the project seemed to have a pay-off in terms 
of involvement. 

We feel this objective was well -attained and our rating is 4.0 to 4.5. 



Objective Six - Develop a program that- will improve ABE instruction * 
while maintaining the lowest possible cost-per-pupil . 

We feel this was an unrealistic objective, especially the aspect 
of "lowest possible cost-per-pupil" . A demonstration project will 
usually not operate as efficiently as, it could. Thus, the cost-per- 
pupil was quite high. Some comparisons are available. A home- 
based ABE program that provided four hours of instruction per week 
for four months to about 100 students had accost per student of about 
$185. A rough allocation of project costs indicated that the cost- 
per-student for the treatment group was about $2,400. It mast be 
recognized of course, that there were some 2,900 othfer persons who 
received some benefit and their cost was about $200 per student. 
Furthermore, these costs would be reduced considerably as the program 
was re-run. Even so the cost-per-pupi 1 was high as would be expected. 

There is no real evidence that the program improved ABE 
instruction. Materials were developed that appear to be useful 
and they were used. Vlhether they were an improvement is not known. 

An objective like this one is nice to have, but, as stated, is 
quite unrealistic. We have to rate the attainment of the objective low, and. 
the rati ng is 1 .0 to 1 .5. , 

Objective feyen - Assist in the development of skills that can lead 
to new careers for home-study aides and other staff members. 

In contrast to Objective Six, this objective was realistic. The 
project was not committed to development of new careers. Rather the 
very realistic intent was that the staff would learn things that 
might lead to new careers. 

Although all of the staff members were considered in the 
objective, it is probably correct to assume that the objective pertained 
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primarily to the home study aides (home visitors). The home visitors 
did receive a considerable amount of training prior to and 
during the demonstration.' Some j'i'^satisfaction with the training was 
expressed by the home visitors. They felt they should have been 
able to function more as a teacher than they di(J. The teaching 
role was not regarded as the proper role by the project, however. 
Consequently, the training was oriented to the home visitor being 
able to establish a helping relationship with the student, and to 
knowledge of resources that were available for the student. The 
home visitors did indicate that they were well-trained in these areas.' 

It is important to note that at the end of the project two of 
the home visitors were hired in one of the counties to do those 
things they had done in the project for an Area Vocational-Technical 
school. Thus, even though there was no job descrip-tion in any agency 
for a person with home visitor skills, in one county they did regard 
this function to be important enough to find a way to create such a jol 

We rate the attainment of this objective 3.5 to 4.0. The 
rating might have been higher, but we judged that there were some 
problems with the training such that the role dermition for the 
home visitor was not as clear as it might have been. 



Objective Eight - Demonstrate involvement of disadvanta ged individuals 
in the development and implementation of such a program. 

The prajfict did solicit and receive input from 'disadvantaged 
adults in the planning and development stages. The inputs were in 
the form of interest surveys, TV viewing interest surveys, and 
reactions to materials. These efforts were important in that they 
did provide good information as a basis for some of the important 
decisions. 

The word "involvement" implies more to us, however, than being 
an information giver. The word connotes things like commitment, 
interaction, and participation in decisions. We don t feel this 
kind of invilvement was evident. The project exhibited a fault we 
feel is present in too many of our educational programs. Typically 
we do a needs assessment (formally or informally) and then we 
interpret the results and decide on a program that will do something 
to or for somebody else. That somebody else is not involved in the 
interpretation and decision making. As a consequence the programjs 
perceived by the participant as "their program" rather than ours . 
We criticize RFD in this sense. Disadvantaged individuals were not 
represented on the advisory committee. Some did sit on a fie d advisory 
board but this board did not seem to have functioned very well. 



ERIC 



•13- 



None sat-in on any of the decision making sessions. Of the participants 
with whom we visited the tone was, "It was nice that the University 
(or government) did this for us", rather than one of really identifying 
with the project as "ours". 

The almanac and the various feedback procedures were probably 
attempts to obtain identification with the project by participants. 
They may have had some effect in this way. Our impressions- based on 
conversations and the obvious resistance to being interviewed by many 
at the end of the project, however, were that the target audience did 
not have a perception of being involved in the project except as a 
recipient of favors. , 

Our rating of attainment of this objective is 3.0 to 3.5.L 

Obj ective Nine - Develop participant skills in the basi c fields of 
^SiUHT^ itTyn and computation while improving the capability of the_ 
tFr"qet~audience to exercise citizenship responsibilities. 

This objective is the critical one for many of the people who 
will judge the P-^D project. The project was funded as a demonstration 
in Adult ^asic education. Therefore, it should give evidence that it 
delivered Adult Basic Education. Unfortunately there is no strong 
evidence that the project met this objective very well. In the 
critique of the design, we have indicated some of the reasons perceived 
by us for the project not attaining this objective. These reasons are 
summarized here. 

1. 




"Develop skills of the target audience in coping with their 
environment while providing the opportunity tojmprove m the 
basic fields of communication and computation." 

In our judgment, even this objective stresses the communication 
and computation areas more than they wereuactually stressed 
in the demonstration. The project reports state that study 
materials were available in the literacy areas, but they 
were rot pushed, and there is little evidence that such 
materials were requested or used. The TV programs generally 
mentioned only the coping skill materials, and the order form 
for materials included nothing in terms of literacy. The ast 
few TV programs did make some mention of the possible availability 
of some literacy materials. In fairness to the project. 
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it should be noted that one reason these materials were 
not mentioned was because of a concern that a large number 
of parents would order them for their children. 

One of the main criticisms of the project that we heard in our 
interviews was that the project erred when it de-emphasized 
the literacy training as much as it did. The point made 
was that while the coping skills are important, the basic 
three R*s are also important, and that the project should have 
persisted with the original objective in this area. We 
feel this criticism has some merit. 

The design used in the evaluation was not especially appropriate 
to the objective. Actually, the main evidence of relevance 
to this objective is from the controlled experiment. On four 
general criterion measure areas (three of which were content- 
oriented) there were no significant differences at the -05 
level between the treatment and control .group means. Recall 
that the treatment difference was having or not having a 
home visitor. There is less evidence on whether receiving 
'and using the materials, and "viewing the TV program had 
an effect. The only evidence is that the treatment group 
received significantly more materials than the control group, 
but still did not score higher on the criterion measures. 

The demonstration as designed did not optimize the results. 
We feel the readiness of the participants in the experiment 
for an educatidnal experience was over-estimated. It seemed 
that the demonstration wa^ nearly half-over before the home 
visitor had gained enough rapport with many of the participants 
to be able to work with them on educational activities. 

The measuring instruments were based on a large number of 
behavioral objectives that were listed during the first year. 
When the emphasis changed, many of the stated behavioral 
objectives were no longer relevant to the content of the 
materials. It should be noted that the difference between 
the means of the treatment and control group was significant 
at less than the .10 level (one-tail) in favor of the treatment 
group on the one measure that was most relevant to the content 
materials. 

Several errors were made in the operation of the <iemonstration. 
More than a few of the participants did not have workable TV 
sets for much of the time and others were on the fringe of the 
viewing area where reception was bad. The content materials 
were not all prepared or the supply printed was not adequate 
so that some of the .participants in the experiment did not 
receive some of the materials or experienced a long delay in 
receiving ordered materials. 
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There were positive outcomes with respect to this objective. 
The response to the TV programs and the content materials was very 
favorable, and there was a generally expressed feeling that they were 
informative. The demand for the content materials was high. In 
fact, this demand caused part of the problem mentioned in point five 
above. The project supply of some of the materials was exhausted before 
it was realized that there was not a supply available for the experi- ' 
ment participants. Finally, the project did succeed in getting educa- 
tional materials into the homes. Furthermore, the degree of success 
was related to the degree of treatment. That is, those with home 
visitors ordered the most materials, those without home visitors but 
who were specifically recruited (control group) ranked second, and the 
rest of the population ranked third. Exposure to education is the 
first element in education, and the project was successful in exposing 
people to educational activities. 

Our rating of attainment on this objective is 2.0 to 2.5. We do 
feel that if the demonstration had run for another 13 weeks this 
objective might well have had a higher attainment level because the 
participants were just getting into educational experiences when the 
project ended. 



Objective Ten - Develop participant skills from present proficiency 
toward eighth grade and twelfth grade equivalency achievement levels . 

There is little evidence that this objective was attained. In fact, 
after the planning year decisions were made this objective was 
essentially dropped. No clear effort was made in the demonstration to 
achieve this objectives 

Our rating of attainment on Objective Ten is 1.0 to 1.5. 



Summary Statement 

Project RFD was an ambitious effort in all of its phases; conceptuali- 
zation, development, and demonstration. It was also a pioneering effort. 
Its objectives were also ambitious, and in our judgment the results 
were what would be expected of a first effort at an ambitious undertaking. 
Many things were done well, much was learned, and benefits did accrue 
to participants. On the other hand mistakes were made, there were 
errors in judgment, and the benefits were not as great as the promises. 
Our ratings of attainment of the objectives were such that we could 
give the project a global rating of 2.8 to 3.2 on a five point scale. 
This rating certainly reflects our feeling that the project was not as 
successful as would have been desired, but it also was not a failure. 
The project might be considered a relatively high risk type because 
of its innovative nature. There was little past history to permit 
strong predictions of its success, and there was 1 ittle. expertise 
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available to give the experience needed to maximize the likelihood 
of success. A high risk project by definition has a high probability 
of failure. Using this line of thinking Project RFD, as a high risk 
project, should be considered quite successful in that it did not fail. 

The project did demonstrate that a multi-media educational 
approach can be put together and delivered into the home. The project 
also demonstrated that the participation rate using such an approach 
is quite high. The project produced materials that were judged to be 
interesting, informative and generalizable. 



The project staff -and others who have been affiliated with the 
project have learned much in the past three years.' We strongly urge 
that they be encouraged and supported in their further efforts to 
refine and revise the model. Television, radio, home study, home 
visitors, all of these media should be exploited -to expand the educational 
opportunities of adults as well as children.,, The staff of Project RFD 
and others who were associated with the project have made fine 
progress in learning how to use these media effectively. It would be 
tragic if that experience and knowledge were not put to use in further 
work in this area. 
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The Television Component 

Television was one of the three primary components of RFD. In 
the first proposal, the conceptualization of the TV component 
seemed to be one of instruction. That is, television was to be 
used as a mediiijn of instruction. The concept changed during the 
planning year, however. 

A field study was made during the planning year, and the 
results indicated that the target audience preferred news and public 
affait-s type p'rogramming and a "magazine" format. Other s-tudies 
had indicated that TV was effective as a medium for developing 
awareness and stimulating interest rather than in-depth instruction. 
The decision was made then" to use a magazine show format that was 
designed to stimulate and inform. Each half-hour program contained 
an average of 25 program segments none of which were longer than 
four minutes. The segments Included presentations by the show 
anchorman, film clips of topics of interest, interviews with stars and 
with local persons who had important information, helpful ffints, 
cooking, and other topics. The second-year report contains a com- 
plete listing of each program. 

Four pilot tapes were made in the first year. These>were 
shown to persons like those in the target audience; Head Start parents, 
students in ABE classes and GEO students. The reaction to the four 
tapes was favorable in terms of interest and their bejng informative. 

Twenty black and white tapes were then produced for the demon- 
stration. Each was one-half hour and each used the magazine format. - 
Duriag the demonstration, one tape was shown four times each week 
over WHA-TV, a UHF station at the University of Wisconsin. The 
air times were consistent each week and seemed to be appropriate for 
maximizing audience coverage. 

An original goal of the project was to develop basic television 
materials that could be used in a variety of rural situations through- 
out the country. This purpose implies that the programs should 
not be specific to one locality. The programs that were produced, 
however, were quite specific. They contained names, dates, addresses, 
and phone numbers in the four-county area. Thus the programs could not 
be used outside of th? area, nor could they be used again en toto in 
the four county area. The rationale for this decision to make the 
programs specific is not clearly defended in the project material. 
Obviously^ it was an attempt to personalize the programs and thus 
stimulate interest. ^ 

This generality-specificity question is very important in terms 
of cost and possib: ' in terms of effects. If a program is general 
enough that it can be shown in a variety of locations or shown repeatedly 
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in the same location, obviously it wiVl have a* lower iJnit cost. 
On the other hand, it may lose in terms of effectiveness. For 
example, reruns of the same program in one locality would be pro- 
blematic with respect to viewer interest. The approach that 'RFD 
seems to advocate is that some program segments can be produced 
for general distribution, but the program should be put together " 
for a specific audience. This approach is probably realistic if 
an optimum balance of cost and effectiveness can be achieved. 
Obviously, there are many questions about TV programming that need 
to be studied. The*d6cision of Project RFD on the generality- 
specificity issue rett-'lected an intuitive answer to a series of , 
questions that might well be studied empirically. We do not fault 
the project for the decision. The decision did, however, result 
in their not meeting one objective, that of producing TV materials 
with a high degree of genera 1 i zabi lity. 

The following are offered as specific points of criticism of 
the TV component. 

1. The programs were viewed by a large proportion of the 
audience. Surveys indicated an estimate of 20% of 
the homes with a TV set on difring the viewing times 

did view an RFD program during the week. Furthermore, the 
proportion of viewers who were target audience was consistent 
with the proportion in the general population throughout 
the demonstration. 

This outcome was achieved even though the- programs 
were in black and white and were shown over a UHF station. 
The exclusive use of a UHF station might be questioned 
because there were sets that did not have the UHF adaptor,- 
and there were parts of the four counties that had poor 
reception of the UHF station. The advantage of UHF was 
programming time and/or cost. If the commercial VHF 
stations had been used, the air time would either have 
been on public service time (poor times usually) or 
been purchased time (high cost). 

2. The reaction to the programs was favorable by those who 
viewed the pilot films and those who viewed the program 

in the demonstration. The programs were generally well-done 
and technically sound. Some specific program points of 
critique are: 

a. The choice of a local well-known personality was 
favorably received. He handled the show well 
and also brought an audience. 

b. The topics were appropriate. There was some feeling 
that there were too many segments in the programs 
which may have been confusing for the target audience. 



c. The star segments were not considered too necessary. 

d. Some of the film clips technically werfe not well 
done, especially the star segments. Other film clips 
had probably been seen before in other programs and 
thus were not unique to these programs. This might 
have damaged the impression that the program was 

• unique to the audience- 

e- The [}roduction, graphics utilization, and writing was 
technically sound but not especially imaginative. 
This is profeibV related to the fact that^the 
programs were produced in a relatively short 
period of time* Creativity and imagination need time. 

^f. The RFD logo and theme music came on very often during 
a program and might have caused viewers to think the 
program had ended, 

g- .The level of abstraction and vocabulary in some segments 
was quite high for the target audience- 

h. The attempt to integrate the content of the TV programs 
with the othfer components was only partly successful- 
There were references in the program to the content 
materials, but one would have to view every program 
to know what was in the content center. Further- 
more, it would be likely that one might soon forget 
the content center reference of an earlier program. 
The Almanac served some purpose of integration between 
the TV and content centers. The integration intent 
^ was only partly achieved. Perhaps strong integration 
requires that the TV programs be produced on a continuing 
basis with several references to materials in different 
contexts. l 
^ i- The TV productions seemed to have been dominated by 
persons trained in TV- The programming might have 
benefitted from more input from persons who were 
knowledgeable about education of undereducated adults. 

The project did demonstrate that TV can be used in a multi- 
media approach, and that undereducated adults will participate 
in educative experiences with this model. The programs are 
not general izable, but the model is. 
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ABOUT ME AND MY MONEY 
PART 1. ABOUT MY HOME 
BUYING GUIDES 



HOME FURNISHINGS AND APPLIANCES, Bulletin 3C4 



Buying Small Appliances 



Do you Know \vf)jt to look for when buying 

M.iny sniiill .jppluinces help u person do 
)uL)s tMsuM ,ifKl fijsler Slop .md think of some 
of X\M' 'pp^'"^'*-^'S you use for making your life 
nior*' n'tiifoHdhjp. You m«iy thif)k of these, 
tojslt'f, (l(Hk, coffeemuker ur iron. Now 
think of ii snwM apptuince you nwy not use 
often, sucti dii electric knife 

There are many srrvill of)pliances you can 
^ buy. Most of them are fairly expensive. Don't 
buy small a()()liances you won't use often. 
Some appliances are used almost every day. 
Others may he used only a few tinKJs a year,: 
Before you buy, decide tiow often you would 
use an appluince. Decide if rt is worth the 
price to you. Sma^I appliances often do things 
thijt major upplunccs do. For example, you' 
nviy <'Mv.idf r bwytng tin electric frying pan.. 
You prob.'bly (Jo most of your cooking and 
t).4int| <;•! yt.Hir sluve Because of this, you 
*r.;y <Jt'tnie <in el<»Liric frying pan really isn't 
nt'L^'sScJf y for you 

KINDS OF SMALL APPLIANCES 

. Thfff are two kinds of small appliances' 
thoM; ifiai qivp heat and those that qive 
power A mixi-r, elfctfic fan or c*in o[)efier 
(jivt- p(r.%t'r An irf)n, coffof maker and toaster 
ijivt-' fif'.it Somr of ttx'se ,i|>pliances will do 
ofily onV thinq such <fs to«isi»tKj t)red(J or 
mo^ .f^ij L(jffe»' OlfuTs will do several jobs An 
^'y.jn.pN' ivth<' ,i|.pliufH-e which Ofjen cans, 
crusf} Ke, and '.htirpfn knives It is(J((M()er to 
buy <»n(' <;(jpli«jn( tt.»it does sevt-rjl thiiujs. 
This IS Uui'. Uuwt'wt, (jnly .f you "ecJ evt-ry 
service iImC -jf^ a(>{»liofH i- will do Qthrrwisc. il 
IS cfjKjfif-r to buy Ih<* <ipplhuue llial does tlie 
one ih.inq yoii nt'i'd 
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Hotne Study Materials Component (Content Centers) 



The first year proposal gave an impression that the home study 
component of RFD would be basic literacy materials presented in a 
correspondence school kind of format. If this was the concept then 
it changed considerably during the planning and demonstration years. 
The changes were quite well justified and the home study concept 
that evolved in the project seems to be a sound approach. 

Several things happened with respect to the home study component 
during the first year. One of the first decisions was that a sequential 
type curriculum was not appropriate for the target audience. Thus, the 
idea of having a person progress through traditional graded materials 
was rejected. Related to this decision was a second one that'the 
materials should be immediately relevant. to the needs, wants, and 
interests of the undereducated adults. The materials should be 
concerned with solving problems of the adult rather than teaching 
literacy skills per se. Of coCirse,°the expectation was that 
the study of relevant materials would also provide literacy skill 
training indirectly. It should^be noted that the term "coping skills" 
was used to describe the aim of the content. A survey was made of a 
sample from the target. audience. The results tended to substantiate 
the decision to orient' the content coping skills. 

A model of the content component concept is in the first year 
report. According to this model, the primary content was directed 
at the coping skills. Materials on literacy skills, computation 

and comnunication skills, were available if the stu^ent_identif_i_ed 

a need for 'or interesjt^jji^wQrJciJig-4n--fehese-areas~. 

. A third decision made during the first year was to develop the 
materials for a reading level of grade four to six. Originally the 
intent was to develop materials so that the reading levels of 0 \ 
to 8 would be covered. This probably would have been done by writing 
the same content at three reading levels like 0-2, 3-5, and 6-8. 
The decision was certainly justified from the practical viewpoint, but 
it did somewhat compromise the Adult Basic Education intent of the 
project. 

Another activity of the first year was developing behavioral 
objectives. A large number of such objectives were identified. We 
are confused about when this task was done or why it was not done 
over. A reading of the objectives in the first year report will 
reveal that a large proportion of them are stated in terms of 
improving literacy skills. Yet the emphasis of the content was 
on coping skills. The list of objectives was used by the internal 
evaluators for constructing the criterion instruments. We are 
concer°ned that the instruments were not as relevant to the project as 
they should have been because of-their being based on an inappropriate 
set of behavioral objectives. 
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After the planning work was completed, the s£aff began work on 
^ developing the materials. The areas covered by tJi'S materials 

are presented in Figure U (The numbers afe of the number of persons, 
' who received each unit.) The development task jyas iB^ge; 76 units. 
^The intent was that the units would be put togipther using materials 
that were -available, and the review' of available materials was 
exhaustive. The review revealed thaft about 45 'of the units required 
that all or part of ±^e material would rfeed to be writT:en by RFD 
staff. A bigger task than was anticipated, and one which created 
problems- 

h writing staff was employed during the first and second years. 
Sofne units v#ere produced during the first year and field tested 
with a sampj? of Head Start parents and ABE students. The reaction 
to the matemefls in the field test was favorable. Some problems 
were encountered with the work of the writing staff. ^Some with , 
whom we talked felt that they did not receive enough direction and 
wasted some time. This lack of direction probably occurred 
because of the unanticipated magnitude of the task' arid, because the 
work was of a relatively innovative nature so that the project was 
feeling its way.' 

Most of the units were assembled in time for^ the demonstration. 
Ten were not done, hdwever,- Before discussing the demonstration, 
however, we Will present some judgments about the cpntent center mater 

The project staf^ indicated that they used three*general criteria 
for selecting or developing materials. The criteria were Under- 
standability, Believabili ty, and Usability. We have judged the 
" materials on these three criteria and the sub-criteria urder each. 
The following table shows our judgments. 



Excellent 



Good 



Fair 



Poor 



A. Understandability 
Clarity 



X 



Simplicity' 
Specificity 
Readabi 1 i ty 



X 



X 



B. Believability 

Identity ^ 
Intentionality and Honesty 
Objectivity 
Reality 



X 
X 

X 



X 



X 



C. Usability ' 



Practicability 
ApplicabilH ty 
Transferrabil i ty 



X 
X 
X 



X 
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III CONTENT CENTER ABOUT ME AND MY MONEY 



EXHI8IT0 
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CONTENT CENTER LISTINGS 





PART 1* 






Moduli A: 


MANAGING MY f:AMILY S MONEV 


88 


Unit 1; 


My IncOmt? 


113 


Unj.! 2 


Familv Spendtng Plan 


yu 


Unit 3 


Uttng My Cr#di! Wi$<?ly 




Unit 4' 


Finitly Security Plfln 




Moduli B: 


THE JOY OF GOOD FOOQ 


710 


Unit 1 


A Choice CasterOle or Slow 


664 


Uni! 2 


Supper on !he Titile on Ttme 


700 


Unit 3. 


Meols fnr a Day 




Unit 4 


Wise Planning * Smart Shoppir^g Good Mfilt 




Moduli C: 


BUYING GUIDES 


213 


Uni! 1- 


How 10 Buy Wisely 


316 


Un»t 2 


Smart Food Shopping 


190 


Unit 3 


How to Buy Clothing 


202 


Unit 4; 


Buy mg Home F urntthingt end Apptiincet 


186 


Unit 5' 


SititfiCtiOn Guarinteed 




Moduli 0: 


HEALTH. SAFETY AND SANITATION 


349 


Unit 1:- 


Emergency First Aid 


349 


Unit 2: 


Home First Aid < 


349 


Unit 3: 


Home SMeiy Plan 


349 


\Jn%\ 4: 


Knowing Signs of lllnati 


349 


Unit 5: 


Home Sanitation 




Moduli £: 


MAKING A HOUSE A HOME 


94 


Unit 1. 


A Man'i Hume is Hts CisMe 


117 


UnM'2: 


Takirig Care of tha Inside 


170 


Unit 3: 


Taking Care of the Outstdr 


118 


Unit 4: 


How 10 OO'lt- Yourself Repairs 




PART 11: 


ABOUT MY WORK 



I CONTENT CE NITER: ABOUT fV»£ 





Module A 


100 


Unit 1 


95 


Un.i 2 . , 


92 




96 


• Unit 4 




KV>dula a 


70 


Unit : 


n 


Unii 2 


65 


Unit 3 


60 


Unii 4 


90 


Unii 5 




Module C 


77 


Unit t 


3b 


Unit 2 


.49 


Unit 3 


69 


Un»» 4 



UNOERSTANDING MYSELF 

Who Am I Accepting a View of Myself 

Who Am I A Positive View of Myself 

What Affects Me Heredity and Envtronmar>t 

What I Believe Developing Character 

MY y^ELL BEING 
My Physical Health 
My Mental Health 

My EducattOn * 
My I pisure and Recreation 
M.iluring and Growing Old Gracefully 

BtCOMiNG A BETTER PERSON 
I Arn Changing How Change Affects Me 
i Am a Person Oignity. Iniagrity 
MotivaiiOr> Self Oirectipn 
i Can Makif Things Charrge 



93 
93 
9v^ 
93 
93 



38 



Module A: 

Unit 1: 
Unit 2* 
Unit 3 
Unit 4 
Unit 5. 

Module B. 

Unit 1. 
unit 2. - 
Unit 3. 
Unit 4: 

Module C: 

Unit 1 
Unit 2 
Unit 3 
Unit 4: 



HOW TO LOOK FOR A JOB 
Sources of Job; 
Lefters and Forms 
Personal Interview 
Important Job Facts 
Summary for Job Hunters 

HOW TO APPLY FOR THAT JOB 

Under development - 

Included tn Module A 



HOW TO GET THE MOST FROM MY JOB 
My Responsibilities to the Job 
My Boss's Responsibilities to Me 
Getting AheiKJ on the Job ^ 
Laws Protecting Workers 



IV CONTENT CENTER: ME AND MY COMMUNITY 











Mc dule A: 


ME AND THE PEOPLE 








28 


Unrt 1 


Of : • < People, 8v The People For The Peopie 


M 


CONTENT CENTER. ABOUT ME AND OTHERS 


37 
41 


Ur.it 2 
Unit 3 


Th» 'Jeclaratton of Independence 
Oiir ConshluhOr. and What tt Means 








36 


Unit 4 


How We Organi/e to do Business m America 




Module A 


MV FAMILY 




Module B. 


MY GOVERNMENT. HOW IT WORKS 


/2 


Unit 1 


Home iS for ramily I ivjng 


168 


Unit 1 


My Community * 


]]? 


Untt ^ 


(ii»T!ing A«r)ftg wiTh One's Marnoge Partner 


m 


Unit 2. 


In My Sl.>i*» 




tJmt 1 


UmjMrslflnrlii'tq Fei'in.g^ of Family Mvmbari 


07 


Unit 3 


In My Country 


1 i'J 




GMi(}>ng Children and Building Character 


63 


Unit 4 


In the Wnrirt 


91 


Umt 5 


f amily Good Times 




Module C: 


CITIZENSHIP IS TAKING PART IN DECISIONS 




Mi>dui« 6 


COMMUNICATING WITH OTHERS 


124 


Unit 1 


jL^y/iqhts as a Citi/en 


bO 


UmiI 1 


My Friends 


122 


Unit 2. 


My Rfspunsihitihes at a Cilifan 


04 


Un.i 2 


My NeiqhhorS 


56 


Unit 3 


Heing an InfOrrnerl Cilifen 


82 


UnM 3 


The Arl of LiSlfning 


* 86 


Unit 4 


Population* PolluliOni ConservaiionI 


66 


Unti 4 


RfSpechng Other People's Ideas and Beliefs 


Module 0: 


WHERE TOGO. WHO TO SEE. WHAT TOGO 




Module C' 


WORKING WITH OTHER PEOPLE 


141 


Unit 1 


Welfare* anrl F miiloyrtuint Services 




Unit 1 


People 1 Deal with lor 5>Hrvices 


141 


Unit 2 


Medical and Health Services 




Un.i 2 


People 1 Work with on tha Job 
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We obviously are impressed with the materials. Furthermore, more 
of our ratings would be in the excellent column if we had judged only 
those materials written by RFD. 

Following are some specific points about the materials 
developed by RFD. 

1. The materials did deal with essential coping skills. 

2. The ideas were presented in interesting fashion for ease of 
reading. 

3. There was excellent use of pictures, charts, and graphs. 

4. There were many good and specific suggestions of ''do's and 
don^ts". 

5. The materials were focused on one important concept at a time. 

6. The approach was appropriately geared to appeal to adults, 
e.g. (a) pictures were of adults,- not children, (b) starting 
units with concrete suggestions rather than abstract principles. 

7. The pictures were of people of varied age and background. 

8. Pronunciation helps' mi*ght have been useful, e.g. lasagna. 

9. A few of the suggestions might be inappropriate, e.g. getting 
Julia Child's Cookbook at the library. ^ 

10. Pamphlet on master mix instructions was not included in 
Module B. How v/as it received? 

11. Are the table setting illustrations appropriate to the 
experiences of the target audience? 

12. We were especially impressed with Modules BiC, and D of 
"About My Home". These were among the first modules completed 
which might lead to some inferences about quality in 
relation to time. 

The, following are reactions to the sel^cte<l materials. 

• 1. The ceading level was generally more difficult and often the 
• , print Was more difficult than ^he "RFD materials. 
2." There was some duplication of content e.g. Module B in 
"About Me" and Module D in "About My Home". 
. 3. Jhe poor reader would need help with many of the words 

and the math examples. 
'4. The selected materials were just not as appropriate as the 
RFD materials, ^hey were a compromise. 



Demonstration 

ks ii.Jicated above, all but 10 units were ready when the 
demonstration started. The intent was that the home study materials 
were to be available to anybody upon request. The potential 
participant was made aware of . the materials in various ways. The 
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main avenue^f awareness were the TV program and, for the treatment 
group, the' home visitor. Other means were the Almanac, action 
line, and personal contact. The participant was sent an order form 
that was very similar in form to the content center listings sheet. 
They were to order those units or modules that they wanted. 

The response was quick and much greater than anticipated. Several 
people ordered everything. These requests were not honored. Rather c 
a contact was made with the person to determine a single unit or 
module that was of most interest, and that was sent. Of course '"^ 
this person was encouraged to submit new orders for single items - 
when ready. 

The large response caused problems. First, the supply of 
materials was exhausted quickly in some of the centers. Consequently, 
there was a lengthy delay in filling some orders. This situation _ 
was especially unfortunate as it affected the experiment. Some 
of the participants in the experiment experienced the delays. We 
feel the project should have set'" aside a complete set eff content 
centers for each person in the experimental and control groups so that 
■^the experiment would not have been confounded by differential non- 
availability of materials. 

A second problem stemming from the response was that the home • 
study teacher did not function as intended. Our interpretation of 
the intended role of this person was one of diagnosis and reaction. 
She was to work with the home visitor or directly with the 
participant in identifying educational experiences that might . 
be appropriate for the participant. For example, she would identify 
materials in the support tracks, computation and communication 
skills, that might be useful for the person to study in order to 
'benefit maximally from the materials in the content centers. She 
was not able to do this. Her time was taken in simply trying to satisfy 
the volume of requests for materials. 

Our main criticism of the home study component is that it 
did not operational ize the conceptual model. Coping skill materials 
did get inlo the homes, but there is little evidence that much of 
the material in the literacy skills tracks did. This was partly 
due to the problem cited above, but also due, we think, to the 
use of such a "soft-sell" approach to literacy skills as not to 
sell them at all. We think, for example, that a literacy skill 
content center might have been included on the order form. AlbO 
the home visitors might have had more training than they did on 
identifying educational problems of participants and suggesting 
literacy material f for study when appropriate. 

We are also critical of the lack of clientele analysis of 
who ordered the home study materials. We do know that the treatment 
.group ordered significantly more materials than the control group 
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or general population. We also know that a greater proportion of . 
'the control group ordered materials than the general population and that 
the quantity is also greater. Needed data are those that describe 
the characteristics of the orderers in the general population. 
Were members of the target audience ordering the materials or 
were the orderers from the middle-class? This information is not 
available. Examination of the figure*^ on orderers on the content 
center listings suggests the pos^sibil ty that the greatest appeal 
was to middle-class women who like to get new recipes and cooking 
hints, " 

Since the demonstration, the project has worked on revising 
and completing the content centers. Several of the modules are^ 
being prepared in the third year for a publisher. The publisher 
will market the materials as part of a new Adult Basic Education 
package. The judgment of the publisher is another bit of evidence 
of the good quality of the materials. 

This arrangement seems rather unique. We have questioned the 
appropriateness of a funded project preparing materials for a 
cormiercial publisher. On the. other hand, materials are disseminated 
well if there is a marketino and distribution organization, and 
commercial publishers have such organizations.'^ With an appropriate 
contract, the commercial publishing route is probably the 
efficient and effective way to get new materials disseminated. 

In surroary, the home otudy materials component was well-conceived. 
There was a tremendous work-load demand in this area, and the demand 
was rather well met. Th^ materials that were developed were of 
high quality and very appropriate for the target audience. The 
component was not able tb fulfill its expectations as well as 
intended, but the staff did learn many things about content and 
operationalizing the condept. Studying the experience of RFD should 
be useful for others contemplating a similar model. 
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Home Visit Component 



In terms of cost-per-uni t, the home visitor component was the 
most expensive aspect of Project RFD. Appropriately, it was also the 
component that was studied most intensively in the evaluation. 

Specification oV the home visit component in terms of numbers 
and function was vague in the first proposal . This came into focus 
during the planning year. It soon became apparent that_tbei^e-vfere^' 
rather strict constraints on the nujflbejrLjof-heme'^isfts . A decision was 
made to restrict the nurnher^of-parfici pants who would receive home 
visits jtD^-a-inarrage^aBTe size. The evaluation personnel were consulted, 
"ciTT^consideritig time, money, and data needs, a figure of 50 was 
determined. The evaluation design was then developed to permit a 
sample of 50 to be drawn from a larger population. These would 
receive the home visits. Another sample of 50 was drawn from the 
population and assigned to a control group of persons who had 
access to all of Project RFD, but the home visits. 

Decisions about the function of the home visitor were also 
made during the first year and early part of the spcond year. 
The idea that the home visitor be an instructor was rejected on 
the basis of the problem of recruiting persons with appropriate 
training and background to the position. It was felt that the 
home visitor must be able to relate readily to .the participant. 
Only after rapport was established could progress be made in the 
educative area. Accordingly the primary functions of the home 
visitor were deemed to bejas adviser, confidante, and friend. This 
decision was studied intensively by the staff, and we feel the decision 
was justified. 

The home visitors were selected and trained during the first 
part of the second year. The selection critenia were quite vague. 
The home visitors were expected to be from an environment where 
they would have experienced intimate contact with people like those 
in the target audience. They were also expected to be articulate, toler- 
ant, and people-oriented. Those hired seemed to meet the criteria. 
It is important to note that all persisted through the project. Eight 
home visitors were selected and each had a load of 6 or 7 participants. 
The home visitors were employed on a half-time basis. 

Three supervisors of home visitors were also employed. The 
^ supervisors were essentially resource people for the home visitors. 
They served as facilitators as well as trainers and helpers. The 
supervisors also administered tests and conducted the follow-up 
interviews. They were employed full-time. 

There is no clear evidence of the numerical adequacy of the home 
visit stuff. Intuitively we have a feeling that one supervisor per five 
to ten home visitors would be sufficient and that one half-time home 
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visitor could work effectively with 10 participants on a weekly 
basis. Some of the home visitors expressed a feeling to us that, they 
could have worked with more people. 

The training of the home visitors was very much based on partici- 
pation and problem solving. A three day session was held in which 
the role was discussed and potential problems identified. Study 
materials were available for the problems and discussions were held. 
After this session, the home visitors were sent into the communities 
to learn about the community and its resources and to identify 
potential participants. The home visitor was to Visit with people in 
power, in agencies, and with potential participants. This experience 
was expected to provide the home visitor with skills in relating to 
various people and with knowledge of the resources. In the process, 
the home visitors identified a population of 300 people in the four 
counties. This population was the group from which the two samples 
of 50 were drawn. 

The home visitors and supervisors met regularly during this time 
and also during the demonstration to discuss their work. These sessions 
also provided training. 

The home visitors were rather critical of the training. They 
expressed.^a feeling that they were not comfortable with their role or 
their knowledge of RFD. They did not feel ready for their work 
in the cjdmmunity. Perhaps part of the problem is that the home 
visitors may have had an expectation that they would be in a teaching 
role. After all, this was an education project. It is also likely 
that the role definition was ambiguous because the project was 
working with a relatively new concept and could not provide a 
clear definition. Although there was discomfort, the home visitors 
did function in the demonstration seemingly as intended and all 
persisted. Furthermore, two were hired in one county by a vocational 
school to function pretty much as they did in the demonstration. 

In the demonstration, the home visitors were assigned to 6 or 7 
participants. They were expected to visit with them on a weekly 
basis. Some were not able to get with the participant this often, 
however. Iney were to function as a friend and provide support, 
advice, help, and encouragement as needed. 'As might have been 
expected, it took some time before the home visitor was accepted in 
this role, in some cases nearly all 20 weeks. Consequently, the 
home visit was not well -integrated with the other components. We 
feel the project was over-optimistic about the immediate effect 
of the home visitor.. Either the participants should have been 
selected on the besii. of readiness for an educational experience, 
or the home visitors should have had some lead time to establish 
rapport before the other components were introduced. We believe that 
this factor was very important in the project's not having a 
deinonstrated effect. 
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The training of the home visitors might also be criticized 
in its de-emphasis on the instructional role. The home visitor might 
have been better prepared to make decisions and suggestions about what 
materials the participant could study with benefit. Also, the 
visitor might have been trained in helping the participant use the 
materials not only for the information but also as a means of improving 
literacy skills • We imagine that the need for this kind of training 
would have become more evident if the demonstration had extended over 
a longer period of time than it did. 

The home visitors were sometimes in an uncomfortable' position 
of being apologizers for the project. They often encountered 
situations where a promised TV repair did not occur, ordered 
materials were not received, or other promised action did not 
materialize. Furthermore, the visitors learned to distrust the 
project somewhat. For example, they thought they were hired 
for 10 months but were terminated after 9 months, and the travel 
expense provision was changed during the project. In general, 
we doubt that the home visitors identified very ciosely with the 
project. It almost seems that they became adversaries in some respects, 
which, if so, likely had adverse effects on the project. 

/jTie^'pr^^ that were encountered with the home visit component 
were understandable. We have little knowledge base for such 
activities, ahd the experience of RFD has contributed to the knowledge 
base. Despite the problems, the home visitors did function effectively. 
They were able to work with the participants and build a meaningful 
supportive relationship. They were also effective in that the 
participants with whom they worked were involved in many kinds of 
educational activities that they likely would not have had without 
the home visits and the rest of RFD» 
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Other Components 

0 

(Action Line, Almanac, Radio) 

These three components were not part of the original conceptuali- 
zation, and only the Action Line component was described in the 
second year report. A brief discussion of each component is presented 
below. 

Radio 

Very little information is available on the radio programs. 
Apparently these were developed as short sp6ts (3 to 5 minutes) and 
were to serve a motivational purpose. Relatively little money or 
time was devoted to this component. 

There is little basis for judging the adequacy of this work. 
Radio does have some unique possibilities that have already been 
demonstrated in things like correspondence study. We would suggest 
that the use of radio in the model be expanded and studied. It 
may well be that radio could be used effectively for motivational 
purposes at a much lower cost than TV. 

Almanac 

The RFD Almanac was a four page newspaper. Six issues were 
printed and distributed to the participants at regular intervals 
during the demonstration. Its content was closely tied to the 
TV programs. Tips, recipes, descriptions and "advertisements" 
about RFD materials were included. Its format was quite attractive, 
and it was very readable. The vocabulary level, however, seemed 
to be somewhat higher than one would expect from the target audience. 

The, rationale for the almanac has not been recorded. From 
our interviews with the staff and our examination of the paper, 
we have decided that the almanac was used as a way of maintaining 
a contact between the project and the participant. It seemed to 
be designed to stimulate interest in RFD and to provide information. 
Reactions to the almanac were quite favorable. 

The RFD Almanac was a very interesting idea. It was relatively 
inexpensive, and our impressions from readinq it were that it 
probably served useful purposes as a stimulator, in maintaining 
interest, and in providing information. The contribution of something 
like an almanac to the success of a home-based model merits further st 

Action Line 



The action line component was designed to provide the learner 
audience with a direct communicative link with RFD. Before action 
line was conceived, the links were via the mails or, for a few, the 
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home visitor. The personalized nature of the telephone communication 
along with the promise of immediate action gave action line a 
unique and important conti ution in the project. 

We wereHmpressed with the careful planning that was used in 
developing and implementing action line- Anybody considering such 
an activity would benefit from reading the description of this 
component in the second-year report of the project. 

Action Line was used. There were some 1600 calls over the 
line in the five months. Fi:rthermore, the satisfaction of the callers 
with the action or information received was high. 

Action Line was relatively inexpensive because it was staffed 
by 50 volunteers from a church organization. The use of volunteers fo 
such an activity is a good approach. We do feel some concern about 
the general izability of this procedure to other communities, and 
with the question of how long the activity could be maintained 
with volunteer help. 

In our judgment the Action Line component made a definite and 
unique contribution to the project. It is a technique or medium that 
would appear to have many possibilities in delivering information 
and stimulating participation. 
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Design and Evaluation 



The RFD Project was fortunate to have the Psychometric Labor- 
atory of the University of Wisconsin School of Education, involved 
with the internal evaluation. 'The Psychometric Laboratory made 
available a number of qualified individuals for assistance in the 
internal evaluation of ((FD. 

The internal evaluation had three distinct aspects: a) the TV 
component, b) the TV and the content material component, and c) the. 
home visit component. Each aspect was evaluated in different 
ways and, in order to reduce complexity, this report will consider 
each evaluation aspect separately. 

The TV Component 

Evaluation of the TV component deal t initially with the problem 
of trying to obtain information about the impact of specific TV 
tapes on the target population. The original plan was to submit 
new tapes to groups similar to the target population and ask for 
a detailed critique. Those participating were to be subdivided into 
a viewing group and a control or non-viewing group and an achievement 
test was to have been given to both groups at the termination of 
the viewing for the experimental group. Failurerto find sufficient 
participants forced a modification of this prq«dure. Ultimately, 
all participants viewed the TV tapes and were asked to indicate 
what they liked or did not like and what was helpful and appro- 
priate. Since the TV tapes were still in production, future 
tapes could reflect consensus views arising from such discussions. 
Generally, the evaluation of the tapes was reported as favorable and 
the procedure was no doubt rich in suggestions for writers and producers 
One wishes, however, that the original plan could have been inaugurated 
since it. might have afforded an early indication of the difficulty 
of materially affecting the achievement of ABE individuals unless the 
educational experience, in addition to being captivating, has 
focus and direction. 

A second concern with respect to the TV component centered around 
the actual population viewing the RFD programs — yJho were they and 
approximately how many were there? Various telephone' procedures, some 
of them quite extensive in nature, were employed during this evaluation. 
There is data that the viewing audience for the RFD audience was 
high — possibly as high as 24 percent for rural viewers and 28 
percent for urban viewers. It is also apparent that the programs 
attracted a much higher percentage of women (80%) than men(20%) and this 
is reflected throughout the RFD program in numerous ways. For^ example, 
all of the home study visitors were women. 
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The TV and the Content Material Component 

Again, the ABE- type participants were aske(^'to respond to 
various of the content materials as they were prepared. These 
surveys were limited in nature and involved no comparison materials. ^ 
It does, however, seem reasonable to conclude, on the basis 
of what was reported, that the content materials were generally 
judged to be relevant, readable and clear by subjects sharing many 
of the concerns and experiences of the target population. 

The Home Visit Component 



Since one of the most costly features of the RFD program 
relates to the home visit component, it is appropriate that most 
of the formal internal evaluation was directed to, this component. The 
Psychometric Laboratory proposed a two-group design involving 100 
subjects as a test of the effectiveness of the home visit component. 
In an early position paper it was forcefully argued against the 
adoption of a pre- and posttest format. Instead, it was decided 
to select 100 subjects all of whgm met certain adult basic 
education criteria and then randomly to assign 50 subjects each to 
an experimental and a control group. The experimental subjects 
were to receive the home visitor while' the control subjects 
were not. Obviously, control subjects had access to all other RFD 
components — TV programs, content materials, hot line, etc. 
Ultimately, 7l' females and 29 males were randomly (except for 
some geographic constraints) assigned to the experimental and control 
groups. Although it was not apparently intended to be this way, 
through some inadvertence the randomization actually resulted in a 
"separation" of some six family members into the two diverse groups. 
This seems to have been a poor procedure in that it affords the 
possibility for results which are not measurable. If one has 
expressed a desire to have the home visitor and only his wife (or 
husband). receives one, what will be his long-term response? Will 
he despair and become indifferent? Will he work harder on the 
content materials which come into the home? Will the selected 
subject "protect" the one not selected by less than a maximum 
effort? There seems to be no- way for assessing such outcomes. 

At the end of 20 weeks there was an extensive evaluation 
involving a comprehensive test which was prepared by the University 
of Wisconsin Psychometric Laboratory. Before this battery is 
considered, it is important to indicate the presence of another 
unfortunate problem in the administration of this test. It relates 
to an initial promise to the participants that no formal evaluation 
would be given. Many subjects were apparently upset by this sudden 
request which actually came sometime after the home visits had 
terminated. In addition, the home visitors, who were called out to 
administer the test, were most unhappy about their particular role 
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in this and thiL unhappiness was intensified in that their tenure 

with RFD had terminated one month before they had expected that 

it wouTd, One cannot, of course, assert that these factors influenced 

the test results. Indeed, it seems, unl ikely that any influence 

of this sort would hav^ exerted a systematic effect. And yet, it was 

not the most advantageous circumstance in which to evaluate the home 

study component 

' Each of the available subjects (46 experimentals and 46 
controls) were given a battery of six tests which was prepared by the 
Psychometric Laboratory, Three of the measures were: 1) communication 
skills, 2) computational skills, and'3) coping skills. All three 
tests were ingenious, readable, and interesting but they did demand 
one hour and thirty nrinutes for completion.. Although there was 
no formal validation procedure for these tests they had high 
reliability and there is no reason, particularly in view of the 
competence of the people of the Psychometric Laboratory, to question 
the quality of the instruments. Three additional subtests dealt 
with: 1) the locus of perceived "control" (internal versus external) 
that a subject experienced, 2) an attitude measure toward education, 
and 3) a "behaviors" test entitled, "The World About Me", All three 
of these latter scales required an additional thirty minutes to complete. 

The results of 'these six tests indicate no significant 
differences between the experimental and control groups* In fact, 
if one invoked a one-tailed comparison (which seems appropriate in 
that a priori predictions were implicit in the entire test procedure), 
there is only one test, "The World About Me", which approaches 
(.08) the arbitrary ,05 level. Even in the^^most ^promising result the 
mean difference between groups was only 2,45 (41.54 against 39.09) 
and one can only question what such a difference means in. practical 
terms, quite apart from any statistical considerations, 

A follow-up interview was conducted with members of the 
experimental and control group some six months after the demon- 
stration. *The purpose of the interview was to determine if there 
were long-term effects of the project. The interviews were conducted 
by the home-visitor supervisors. No differences were observed 
between the groups on the questions. These interviews were resgsted 
by several participants. They did not expect them, and they were 
suspicious of the purpose. This projeict certainly demonstrated 
the importance of the problem of sensitivity of the undereducated 
adult to data-gathering procedures. 

The objective results of the home study component are most 
discouraging. On the other hand, it is important to consider that 
the time frame of the program was unusually short. This is particularly 
the case if one considers the initial problems the home visitors 
had in gaining the confidence of the experimental subjects. They 
began their contacts as friends and only gradually did they shift 
the nature of the visits to work sessions involving various 



content materials. In many cases there simply wasn^t enough time 
to get the study program going effectively- The external evaluation 
team was -able to observe considerable progress with some subjects 
in one county where two home study visitors were retained by the 
County and continued to work with some of their subjects long after RFD 
was terminated. It is quite possible that a twenty week period 
is necessary just to ensure that the hom& study visitor can be 
effective in a more explicit educationaWprogram, 

It should also be pointed out that the skills tested in ,the 
Wisconsin Test of Adult Basic Education (WITABE) were general in nature 
Even had tfie home study component been more focused on adult 
basic education, it is difficult to see how life-long patterns 
of dealing with reality could have been materially altered in so 
short a period. Another way of discussing the WITABE ,is to say that 
it was most certainly an instrument which afforded little oppor- 
tunity for manipulation by direct ''coaching^' of participants. All 
of this reflects crpdit to the RFD. staff and to the internal 
evaluation team who. insisted that their program be given an honest 
appraisal, / 

One of the project's consultants, Professor Robert D, Boyd, 
made an important/' point in the RFD Newsletter (August 1971) when 
he said that RFD /might not significantly affect basic reading ability 
or other narrow Educational objectives but might still prove to 
be a powerful program if ft resulted in differences in self-esteem 
and community iwolvement. Unfortunately, the objective evidence 
does not supporll a contention that such changes took place. But 
once agairf tf>e Brevity of the program has to be a consideration. 
There is evidenie secured by the external evalfiation team that 
experimental participants requested, on their own initiative, " 
significantly more content materials than did control subjects or 
a group of subjects who were randomly selected from those who 
had made at least one request. This seems to be the kind of behavior 
Professor Boydjhad in mind and this particular result should not 
be dismissed as trivial. It js unfortunate that some unobtrusive 
measures of actual behavior could not have been secured on experimental 
and controf subjects such^ things as their frequency of use 
of the Public »lil>rary, their' understanding and use^of community 
agencies, etc. The procedures used in selecting experimental and 
control subjects could in itself have served to sensitize control 
subjects with respect. to educational attitudes and have militated 
against differences in some of the measures/ At this point it 
would have been advantageous, althoush admittedly costly and diffi- 
cult, to have secured test results from a second control group 
consisting of subjects meeting the adult basic educatio.n criteria 
and requesting at least some RFD materials. 

One other point should be made from the finding regarding 
participation. The^ results reinforce research that indicates that an 
overt qommitment to* doing something is reltfted to participation and 
persistende. ' ^ ^ 



One final evaluation of the hpme visit component was ■aa extensive 
"debriefing" of the home visitors by the staff of , -the P^chometric 
Laboratory. This report, which parallels many of the things the 
external evaluation teaifi encountered in their talks with available 
home visitors, points to perceived inadequacies in the initial 
training of the home visitors. There were other complaints which 
grew out of the role-arabiguity home visitors felt as "teachers" 
but, more basically, as friends of the participants. Prob-lems or 
difficulties of the home visitors resulted from faulty communication 
(e.g.', the p.reviously discussed mixup with respect to whether or 
not subjects would ultimately be evaluated) and a failure of the RFD 
staff to^act promptly on sorfle oftheir initial commitments. 

The following are offered aT^specific points of critique of 
the design and its execution. ^ 

1. The design was appropriate for determining the effects of 
the home-visit component. A problem with the design, 
however, is that it provides little opportunity to assess 
the e^'ects of the TV and content materials. Some^sort 
of indicator of change would have been useful. ^ 

2. The evaluation work was kept separated from the operation to 
maintain objectivity. A problem with this procedure, 
however, is that the evaluators were not cognizant of some 

of the changes of emphasis, e.g. reduced emphasis on literacy 
skills. 

3. Many data were gathered and analyzed. It would'have been 
desirable if there had been more data on characteristics 
of those who ordered content materials and the users of 
action line. 

4. The operation of the demonstration was faulty in that some 
homes did not have working TV sets, some participants were in 
a marginal viewing location for the station, and the content 
center materials were either not done or in inadequate 

■ supply so that the participants failed to get what ms 
ordered or experienced a long delay. 

It is easier to criticize than create. Generally we were favorably 
impressed with the design and execution of the demonstration and evalu- 
ation. We do feel it important, however, to point out some of the 
possible reasons for the lack of impressive results. 
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Dissemination 

Project RFD was very effective in its publicity and dissemin- 
ation activities. We have heard comments like "RFD sure tells 
its story", "They really tell what they are doing", etc. from people 
all over the nation. The fact that the project director had a 
background in pub ^'city work was a factor in determining the amount 
and kind of publicity and dissemination used. We were 'impressed with 
this work. The quality was good and the techniques were quite sophis- 
ticated. Educational developers would benefit from studying and using 
the dissemination techniques of RFD. 

Of course there was some criticism of the dissemination activities. 
Some said that the project "tooted its own horn" and "I get tired 
of hearing about RFD". Some of these comments were probably justified, 
in other cases there seemed to be an element of professional jealousy 
in the remarks. Certainly there is a line where publicity has 
overkill possibility. Most educational activities are nowhere near i 
this line. Our impression is that RFD did publicize itself more 
than most educational programs. The publicity was in good taste 
and honest, and we feel the educational community would benefit if 
most projects told their story as well as RFD has. The'^ following 
material discusses the internal and external dissemination efforts 
separately. 

Several techniques were used to nublicize the project to the 
potential participant as well as other persons in the area who might 
be interested'or affected. Among the techniques were the following: 

1. Several meetings were held with agency people in the 
^counties, i.e. welfare, extension, adult education, etc. , 

to explain the purposes artd procedures of the project. 
These meetings helped to forestall conflict when the demon- 
stration was done, and alsa enlisted cooperation from the ; 
agencies. 

2. Booths for the RFD Project were set-up at county fairs. Jhkse 
booths made persons aware of the project, and also solicited 

a large number of names of people as potential participants. 
' ' , f 

3. Shortly befor ttie demonstration was started, the project took 
advertisements in papers, radio, and TV. It seemed this" 
advertisiTio paid-off in that the^a was a relatively large 
irmiediate audience for thft TV programs and a large volcime 

of requests for materials. 

4. The almanac, critiqued elsewh6y4 in this report, was also 

used to disseminate information about the project to participants. 
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5. Members of the project staff spoke to several groups, like 
/service clubs, about the purposes and procsdures of the project • 

6. Surveys were made of adult interests and TV viewing habits, 
and meetings were arranged for feedback about the TV tapes and 
the materials. Dissemination was an important by-product of 
these activities. 

The RFD Project regarded their program as a product to be 
marketed, and they did an effective job of marketing. The effective- 
ness is demonstrated by a high participation rate by the adults in the 
four county area. 



External 

The project also did many things to disseminate in ^mation 
about the project to audiences around the country. Among these 
were the following: 

K A newsletter was published monthly and distributed widely. 
The newsletter was attractive, well-edited, and informative- 
It was also honest in that it informed people about what had 
happened and of the problems. 

2. The project staff made presentations at professional fneetings 
and served as consultants to various projects. Some of the 
staff are now consulting with the Adult Learning Program 
Series (ALPS) project. The impact of these activities is 
demonstrated by many letters in the files from persons all over 
the nation in which high interest is expressed in RFD. The 
experience and knowledge of the staff of RFD are also, permitting 
the contribution of many ideas to the ALPS project. 

3. The project staff solicited for expansion of the project to 
the Central States region. The intent was to develop a network 
in the several states and provide RFD to the undereducated 
adults in a larqe region. The proposal was not supported by 
the ABE persoMPcl in those states. Thus, this effort at 
dissemination failed. 

The project staff also attempted to interest the university 
and various state agencies in Wisconsin in carrying on 
with the work. Some interest was ^expressed, but nobody did 
make a commitment. The stated reason was that budget cutbacks 
- prevented the university or the state agencies from adding 
an activity. 

These are important events that.have to be regarded as negative 
outcomes. The budget cuts were a factor, but it is also 
possible that the people vteYe not convinced that RFD had 
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demonstrated a sufficient level of effectiveness to justify 
its becoming operational on a broad base. We tend to feel 
the decisions were justified. However, this statement 
should not be interpreted that we feel RFD is a failure. 
Rather we feel that the model has promise, but that it 
can be improved and should be before it is considered ready 
in an operational sense. 

4. The third year of the project is essentially a dissemination 

year. Project staff members are still presenting to professional 
groups and consulting. There are two other kinds of 
dissemination activities this year. First, the project 
has arranged for and is making formal presentations about 
the project in the ten USOE regions. Interested people from 
the region are invited to attend these two-day meetings and 
receive the story of RFD. About 300 persons attended 
these meetings. We attended one of the meetings. The presenta- 
tion was interesting and honest. 

There is one concern we have about these meetings and the 
third-year work generally. Evaluative information should 
be available for the dissemination presentations. Unfortunately 
much of the evaluative information was not available until 
the middle of the third year and some were only available by ■. 
the tenth month. We would criticize the project in its not 
pushing harder to get the evaluative information and analyses 
done quickly so that these kinds of information would have 
been available for most of the dissemination year. 

The second activity of the third year is to revise old and also 
write new content materials for publication. A publishing house 
has contracted to publish and market much of the content material 
of RFD. This should lead to widespread dissemination of these 
materials . 

The third year of the project is viewed with mixed emotions by the 
evaluation team. Essentially there are three activities: dissemination 
to regions ahd other meetings, proposal writing, and the preparation of 
materials for publication. We find it somewhat difficult to justify 
the level of funding that the project had for this year. On the other hand, 
we predict that this project will be well -disseminated in terms of ir. jr- 
mation and products. It will not be lost as a dust-covered report on a ■ 
shelf. Perhaps the kind of activity that was permitted in the third year 
is important for obtaining maximum benefit from a project. People will 
be stimulated by the ideas, they will benefit from the availability of the 
materials, and the field will benefit from the fact that the staff has 
had time to reflect and develop ideas for further study of the model. We 
raise the issue without being able to make the judgment. 
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Administration of Project RFD 



Project RFD was administered in the Communications Department of 
the University of Wisconsin-Extension. The project was directly 
accountable to the General Manager of WHA-TV, the University 
television station. Several people on the University staff were 
interviewed. Generally, these people were quite familiar with the 
project and interested in its results. The general feeling seemed 
to be that the RFD model might be very useful for extension activities 
of the University. There was very little crit.cism of the project. « 
One point of criticism, however, was that the project was identified 
as a TV project by its being administered under the TV station. 
There was some feeling expressed that the project should have had 
a closer relationship with the educational programs in the University. 
We see some justification to. this criticism, if for no other reason 
than public relations. The image of the project for many was as 
a TV program producer rather than an education project. Should RFD 
continue in some form, it may be well to consider having it admin- 
istered in a department like continuing education and contracting for 
services with the TV station. Our general impression of the adminis- 
tration of the project in the University is that it was able to 
operate well in and received reasonable support f>^om the University. 

The internal administration of the project was also sound. The 
project director seemed to be running things, but at, the same time 
delegated responsibility in a real sense to the two associate directors. 
An associate director left after "the first year which probably affected 
'= the continuity of the project. A replacement was hired immediately, 
however. The general turn-over of project staff was quite low 
especially for a project on "soft money". 

The fiscal aspects of the project were well-controlled. Apparently 
the budgets were accurately determined so that the budgeted and 
actual expenditures were very consistent. 

In our judgment the project's internal administration was good. 
The following points of criticism are offered as things which, in , 
hindsight, might have improved the project in an administrative sense. 

As indicated above, the image of the project might have been \ 
improved for some if there had been a co-director in adult 
education or if the project had been administered in an education 
department. 

There was some criticism that the advisory council was 
treated in somewhat of a cavalier manner. At times the 
advisory council seemed to be a sounding board rather than 
a group who gave input into decisions. We would also offer 
the critique that the project should have solicited more 
involvement from the target audience. 



1. 

2. 
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It must be recognized, however, that aL some point decisions 
must be made- The staff did make and accept responsibility 
for the decisions, 

3. There seemed to be some lack of coordination among the 
various components. This is reflected by the lack of 
integration among the components- Also the fact that the 
demor^Jtration experienced procedural problems, such as 
nsyVJ in some homes and shortages or lack of home study 
^..---^''^'materials, suggests some coordination difficulties. Some 
of the staff of the ^jroject indicated that they were not 
always sure of what they were doing. Some of this feeling is 
understandable in that the project was in a sense feeling 
its way. On the other hand, there is some evidence of a 
lack of coordination among the project administrative staff - 

These points of criticism are possible problems that we perceived 
in the project administration. In the main, however, we feel the 
project was well administered. 
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Cost Analyses 



Decision making and resource allocation in the private sector 
of the economy is guided, at least in a general sense, through 
market-established prices. By studying the.conditions under which 
market prices are established, economists have been able to draw some 
tentative (and often very restrictive) conclus-^ons about the efficient 
allocation of resources in the private sector. On the basis of 
that information, public policy is formulated for the purpose of 
achieving economic goals, one of which is economic efficiency. 

The contribution of economics to the study of choice problems 
is a derivation from the central "scarcity" premise of economic 
problems. There are some (chiefly the radicals) who argue today 
that affluence— not scarcity— is not the central economic problem. 
Such persons are fond of eloquently written passages such as the 
following description of a "new order" penned by John Maynard 
Keynes some four decades ago. 

We shall honour those who can teach us how to pluck the 
hour and the day virtously and well, the delightful people 
the lilies of the field who toil not neither do they^^rpin.' 

There is much about this type of world that appeals, to all of 
mankind, for who among us can condemn motherhood, the flag, and 
apple. pie? Sadly enough, however, such a philosophy is a tenet of 
religious faith rather than a useful appraisal and contention about 
the real world today. A fuller reading of Lord Keynes reveals that 
even he recognized this harsh >^eality in the following admonition: 

• 

But Beware: The time for all this is not yet. For at 
least another hundred years we must pretend to ourselves 
and to everyone that fair is foul and foul is fair; for 
foul is*useful and fair is not/ 

Like it or not economic scarcity and the efficiency diety are, indeed, 
our gods for a little longer stilll 

Increasing participation by all levels of government in the 
economic affairs of society raises difficult issues. While economists 
have developed an intricate theory of resource allocation for the 
private sector of the market economy very little progress has been 
made in improving our understanding of the process of allocating 
resources in the public sector where the price system plays a 
muted role at best. Roland N. McKean, Professor of Economics at the 
University of Virginia, develops the thesis that an "unseen hand" driven 
by power and bargaining guides resource allocation in tl;ie public sector. 
The author*s cogent plea for greater attention to problems of choice 
and resource allocation are as applicable to the rapidly growing adult 
education industry as to other categories of government. 



Ij.M. Keynes, Essays in Persuasion (New York: Harcourt, Brace 
and Janovich, 1932), pp. 360, 362. 
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All of this is not to suggest that cost effectiveness analysis 
of behavior related human resources research such as project RFD 
is a simple task which answers all questions conclusively. Lest the 
reader forget, the following limitations must be kept uppermost in 
mind: 

1. Economic analysis and the efficiency diety may mask many 
of the nonmonetary considerations which motivate private 
behavior as well as social goals prompting investment in 
education. Treating people as "capital" to be produced may 
distort a useful and purposeful dimension of life, and in 
any event does not necessarily represent the multiple 
objectives which characterize educational investments. 

2. Investment in education and particularly in adults is 
only one of many interdependent ends which increase 
economic welfare. Multicollinearity between independent 
variables such as motivation, customs, socio-economic 
status, and ability, precludes accurate identification of the 
precise contribution of education much less the marginal 
returns generated by education. 

3. Selection of a "proper" discount rate or the "relevant" 
rate of return is ambiguous— a problem particularly 
acute where the social rate of discount is needed. 

4 Quantification of social, indirect, and nonecpnomic values 
is almost impossible. Educational quality, poverty and 
slums, income redistribution, or an alteration in national de- 
fense posture due to greater emphasis on education. 

5. Uncertainty prevails in terms of future benefits, costs, 
interest rates, and time periods in both the private and 
social context. The relative attractiveness of learning 

may change over time as technological conditions are altered. 

Disadvantaged Americans may invest less in education because 

of a higher time preference for money due in part to different 
perceptions of risk and uncertainty. 

6. Unlike investments in physical capital, human capital 
expenditures represent consumption as well as investment 
outlays, again in an unknown magnitude. 

7. Different forms of human capital investment provide for 
more or less mobility and liquidity. The depreciation 
characteristics of human resources also may be important to 
investment in education, yet these relations are seldom 
dealt with. 
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The following pages contain various economic analyses of 
Project RFD. The reader should keep the above limitations in mind 
while considering the results. 

The first part presents an analysis of the budget, and some^ 
cost/benefit considerations are presented in the last pa'^t of this 
section. 

Aggregate Budget AnalysisV 

Project expenditures over the three-year contract period 
totaled $707,793. Of this amount $653,393 was directly charged to 
the RFD project and its operations, the remainder representing 
indirect University overhead charges.-' 

All budget data were compiled and cross classified by 
( a ^ yea ir * 

(b) project media component- (television, home visit activites, 
materials (largely printed), and "other"-primarily 
general project supervision; 

(c) class of service expenditure (e.g., personnel, travel, 
communication costs, supplies, equipment, rental and 
fees, and evaluation charges); 

(d) type of costs, where type (I) denotes basic R and D outlays 
associated with creation and conceptualization of the RFD 
format and delivery system; type (11) denotes all costs 
attributable to the development of materials and programs; 
and type (III) represents expenses Incurred in the operation 
and actual delivery of all RFD components. Table I and 
Figure I summarize these expenditures by relev?nt category. 

In terms of budget allocations by media , expenses associated with 
the development of materials and for thrteTevision operation accounted 
for over t£o-thirds of direct RFD expenditures ($256,000 and $172,000 
resoectively).'* Approximately two- thirds of all costs for these 
two media are attributable to personnel ($179,000 trials, and 

$79,000 for television). Supplies ($39,000 for materials and $40,000 
for television) and rental and fees ($25,000 for materials and $37,000 
for television) account for much of the remaining expenditures. In 
examing budget outlays by class of service, personnel costs loom 
very large, amounting to SWT^SOO of the total direct costs of $653,000, 
while expenditures for supplies and rentals and fees represent a 
much smaller but still significant portion of the total amount of 
funds available. The data in Table 2 depict expenditures and percentage 
breakdowns, again excluding the general overhead rate of 8 percent 
of charges at the University of Wisconsin. 



^General overhead charges by the University of Wisconsin totaled 
$54 OOU All such overhead charges were "allocated" to class of service, 
type of 'media, and type of expenditure in proportion to the percentage 
of the total budget allocable to each activity. 

O *A11 data exclude University overhead and are rounded in this summary. 
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Table I 
Budget Data 

Television Home Visuals Materials Other Sub Total Overhead 



Personnel 
1. 
2. 
3. 


$ 23,324 
55,258 - 

>1 CO 


$ 12,057 
37,661 
4o,Uoy 


$ 37,647 
140,991 
791 


$ 11 ,739=$ 84,767 
51,320= 285,230 
-0- = 44,338 






179.425 


63,1559 


414,335 +34,490=$44B,BiJb 


Travel 
1. 
2. 
3. 


15 
3,156 
-0- 


502 
972 

4 ,y 1 0 


274 
1,999 


2,035= 
8,704= 
-0- = 


2,826 
14,831 
5,596 




O ) i / 1 


6 387 


2,956 


10,739 


23,253' + 1 ,958= 25,211 


Communications 
1. 
2. 
3. 


-0- 
333 


-0- 
-0- 

1 , 1 


-0- 
333 


606= 
5,450= 
-0-= 


606 
6,116 
4,047 








333 


6,056 


10,769 + 898= 11,667 


Supplies 
1. 
2. 
3. 


3,071 
35,138 

1 7/17 

1 ,74/ 


27 
1,002 


285 
8,833 


127= 
22,758= 
-0- = 


3,510 
67,731 
32,195 




39,956 








103 436 + 8.595=112,031 


Equipment 
1. 
2. 
3. 


-0- 
981 
-0- 


-0- 
-0- 
-0- 


-0- 
-0- 

-U- 


209= 
7,608= 
-0- = 


209 
8,589 
-0- 




yo 1 




-0- 


7,817 


8,798 + 734= 9,532 


Rent, Fees, Etc 
1. 
2. 
3. 


-0- 
11,864 
25,224 


-0- 
275 
-0- 


-0- 

222 
24,938 


-0- = 
2,752= 
-0- = 


-0- 
15,113 
50,162 




37,068 




25,160 


2,752 


6S,275 + 5,435= 70,710 


ilaluation 
1. 
2. 
3. 


1,290 
7,204 
520 


1 ,290 
7,204 
924 


1 ,290 
7,285 
520 


-0- = 
-0- = 
-0- = 


3,870 
21,693 
1 ,964 




9,014 


9,418 


9,095 


-0- 


27,527 + 2,290= 29,817 


SUB TOTAL = 


172,476 


111,636 


255,973 


113,308 


653.393 +54,400=707,793 


University 
Overhead 


14,362 


9,302 


21,325 


9,411 


54,400 


GRAND TOTAL 


$ 186,838 


$ 120,938 


$ 277,298 


$ 122,719 $707,793 



Category 1 is Cost and expenses associated with basic research and creation activities 
in the conceptualization of RFD delivery systems. 

Category 2 is Cost and expenses (post conceptualization) of the development of 
materials and programs up to delivery phase. - , . .. . 

Category 3 . is Cost and expense incurred in the operation and actual delivery or 
RFD systems. 
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$200,000- 



University 
Overhead 



TV 



Home Materials Other 
Visits 



$400,000- 



200.000- 



Personnel Supplies Rental All 

a Fees Other 



$400,000- 



200.000- 



R 8 D Development Delivery 
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Table 2 

Suimary of Various Cost Categories for RFD Project 

Project Media Class of Service Type of Costs 
$(000) ~ir~ Wm ~% $(000) % 

Television $172.5 26.4% 

Home Visits 111.6 17.1 

Materials 256.0 39.2 

Other 113.3 17.3 

Personnel 
Travel. ' 
Communication 

Supplies I 
Equipment 
Rent/Fees 

Evaluation ^ 
I: R & D $ 95.8 14.5% 

II; Development 419.8 64.3 

III: Delivery 138.3 21 .2 

$653.4 100.0% $653.4 100.0% $653.4 100.0% 

SOURCE: Table 1 ^ , . 



$414.3 


63.5% 


23.3 


3.5 


10.8 


1.6 


103.4 


15.9 


8.8 


1.3 


65.3 


9.9 


27.5 


4.3 



Table 2 also includes data on the proportion of costs (exclusive 
of overhead) due to basic R&D (type 1), development (type 2) and 
delivery (type 3) activities. Among the potential uses for such 
information is that it can be useful in approximating the minimum 
scale of an operation at which a project breaks even in the seri^ 
that returns equal costs. That is, one can identify the minimum 
return and the project size best suited to an effort such as that 
undertaken by l]!FD. 

Tables 3, 4, and 5 break down and surmiarize the budget for RFD<^^ 
by media and class of service expenditure. Not inc^fided in this 
comparison is a breakdown of the category "other" for it largely 
represents general project management costs and not a form of media. 
As noted earlier, personnel costs are a major share of total costs, 
particularly for the Home Visit program, while supplies and rentals 
and fees are also large' for the television and materials media used. 
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Table 3 
Television Budget 



Personnel 
Travel 

Communication 
Supplies 
Equipment 
Rental and Fees 
Evaluation 

SUBTOTAL 

University Overhead 
TOTAL 



$79.04^ 
3.17 
3.23 
39.96 
.98 
37.09 
9.01 

172.48 

14.36 

$186.84 



45.8% 
1.8 
1.9 

23.2 
.6 

21.5 
5.2= 



100.0% 



Personnel 
Travel 

Communication 
Supplies 
Equipment 
Rental and Fees 
Evaluation 

SUBTOTAL 

Uhiversity Overhead 
TOTAL 



Table 4 
Home Visits 



$92.80** 
6.39 
1.15 
1.60 
0.00 
' .28 
9.42 

$111.64 

9.30 

$120.94 



83.2% 
5.7 
1.1 
1.4 
0.0 
.2 



100.0% 



Personnel 
Travel 

Communication 
Suppl ies 
Equipment 
Rental and Fees 
Evaluation 

SUBTOTAL 

University Overhead 
TOTAL , 



Table 5 
Materials 



- -$179.43'* 
2.96 
.33 
39.00 
0.00 
25.16 
9.09 

$255.97 

21.33 

277.30 



70.1% 
1.2 
.1 

15.2 
0.0 
9.8 
3.6 



100.0% 



a. In thousands of dollars 
SOURCE': Table 1 
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Potentjal ::Benefitsl and Market Charao^teri sties 

Adult edacation efforts such as RFD can make vast socio- 
economic contributions to a variety/of persons, particularly dis- 
'ra^aqed Sericans with eniployment, income, and cultura problems. 
SnSiv anoroximatelv 1 out of 8 Americans are officially (and 
ISI tivel?) de? ne^as members of the poverty population, and 
a like (or greater) population of Americans have 8 or fewer years 
f fnLil .Sool No more than one-third are 65 years or over and, 
al e ewUe. f ilure Tmeet adult education needs contributes to 
the hard- wre of unemployed, underemployed, and poor citizens Tte 
s ze of tMs adult mafket for RFD type ^09^^,^%^^^^^ jograp ically 
and by the definition used, yet it is clear that, out of a popu- 
lation of some 82,000 family units (the approximate size of RF^ s 
market) with a sizable number concentrated in rural areas (some 
24 OoS fami ies) and where income and employment problems are pro- 
portionally high, there is a clear need for project RFD. 

Continuing education of this sort may be of particular importance 
since it i?fers a second chance for adults who once may have been 
adolescent dropouts and only later developed the "jotivation and 
matuHty to acquire the training required in a modern society and 
ZlZm^ Considering coping problems as ranging across such diverse 
lZl7ts as c u r n^g knowledge about consumer installment credit; 
D^oJiding home repair services; learning the basics of adequate 
ffitSnal and health needs; or managing personal finances and 
?ind niSut about alternative job or occupational opportunities, 
II is ?ather easy to envision a family unit benefiting by several - 
I A A Ani\Jl>Ti vear over their remaining lifetime. Further 
•-rD nover- b efifs mara s accrue to children, dependents, or 
other adults associated with an affected family unit, in which 
^c 'e sub tantial multiplier impacts .may spin out of Programs 

u ^-^,r^c« RFn Final! V society itself will, in all pro- 
Tab iftv caotSre sSbsta t benefits'in a variety of ways including 
^ mill informed citifenry and electorate; better health and produc- 
^Tt^, ad To s • f;'lSw'er welfare burdens juvenile^^delinquency or 
crime and fewer edJcational and societal "dropouts." 



Economic research efforl^s have, in spite of numerous benefit 
nuantifiJat on Troblems, identified the desir^abil i ty. of a variety of 
Tb p S t rf r^^^ Presuming that .the cost-effectiveness 

«ln-iale^t of $400 per year in material improvements in coping 
!^hl^ior Je?e forthcoming from an effort such as RFD over a 20-year 
mel me. e^ch ^percent'o^ all family units in the market for such 
ise^lice would receive $4,600 in ascertainable benefits. 

One way in which intangible and"difficult-to-quantify benefits 
or "effecuieness" can be assessed and imputed is to vary certain 
data Daraneters about which uncertainty exists. Suppose that (a) 
h!npf??^ orpvail i.n the $300 to $500 annual amounts as denoted 
Tj2Te 6; (b) success fatios ranged from 1 out of 10 to 1 out of 30 
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families; (c) 3,000 families were exposed to an effort such as 
project RFD; (d) there 'were no social benefits nor any spillover 
or multiplier effects; and (e) the social discount rate is 6 
percent over a 20-year lifetime (in reflection of the public 
sector's opportuni^-y cost of capital). Cost-effectiveness felation- 
ships for annual outlays are shown .in Table 6. 

Table 6 

Hypothetical Cost-Benefit Comparisons for RFD* 

Annual Value " Proportion of Persons Exposed and Materially 

of Benefits Aided out of 3,000 Exposed Units 

$200 

pFesent Value 

Ratia B/C if C = $100,000 
Ratio B/C if C = $200,000 
$300 

Present Value 

Ratio B/C if C = $100,000 
Ratio B/C if C = $200,000 



N = 1/10" 


N = 1/20 


}\ = 1/30 


$690,000 
.■ 6.9 , 
3.5 


$345,000 
3.5 
1 .8 


$235,000 
2.4 
1.2 


$1,035,000 
10.4 
5.2 


517,500 
5.2 
2.6 


345,000 
3.5 
1.8 


$1 ,380,000 
13.8 
6.9 


690,000 
6.9 
3.5 


460,000 
4.6 
2.3 


$1,725,000 
17.3 
8.7 


862,500 
8.6 
4.3 


575,000 
5.8 
2.9 



Present Value - 

Ratio B/C if C = $100,000 
Ratio B/C if C = $200,000 
$500 

Present Value .... I . 
Ratio B/C if C = $100,000 
' Ratio IB/C if C = $200,000 

j 
! 

♦The Annuity value of ll .47 applies at 6 percent for 20 years (see page 52 ). 
1 ■ . f 

Under the stringent assumption that annu^ depreciable 
project costs equal $200,000, benefits are valued at $200 annually, 
and the success ratio is a mere 3.3 percent of the exposed popu- 
lation, the hypothetical RFD payoff would be B/C = $235,000 = $1.17 

$200,000 

for each $1 in costs. At the other extreme of $100,000 in annual 
costs, $500 in annual benefits, and a 10 percent success ratio, B/C = 
$17.25 for each $1 in outlay. The "true" level of effectiveness no 
<loubt lies between such extremes, however, it must also be remembered 
that the above data comparisons assume that all expenditures are 
directly output-oriented. This, of course, is much too restrictive a 
presumption for RFD since some rather substantial proportion of the 
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$708,000 in expenditures was related to demonstration and Experimental 
goals. Then, too, the above comparisons fail to disaggregate 
effective costs and values emanating from various RFD m?aia| 
directed at differentiated markets (e.g., target groups vs.' other 
exposed populations). 

Analysis of Impact- 

Table 6 and the discussion suggest that the benefit/cost ratio 
for a project like RFD is quite ^ble if one can assume at 
least $200 worth of annual benefit- Jnfortunately the evaluation 
data of the project do not support the assumption. On the other 
hand, there are likely many benefits or outcomes of the project 
that were never obsarved, but which were real and could easily 
add'to $200 ne*' year. Repairing something, improved buying proce- 
dures, savi . interest rates, saving a life; these and other 
things are ^, .c:fits that reasonably mighv. accrue to pr9ject participant: 

An attempt was made to quantify the impact of the project 
using a probability modeling techni^'e. The rationale for this 
technique is presented below. 

A Framework for Probability Modeling . 

Management scientists have developed and applied a vast assort- 
ment of analytical models to many areas of managerial control in the 
private sector in recent years. Some examples are inventory 
management, brand preference analysis, credit rating terms, fore- _ 
casting and evaluating aggregate demand for a firm, financial planning 
and investment control systems, and risk analysis of collective 
bargaining and capital budgeting problems. The methodology and 
models used by private enterprise must be restructured to various 
degrees to fit the public sector management context. There are 
three fundamental features which characterize such decision problems. 

1. Probabilities must be non-negative and also equal to one 
for all events which are certain to occur. 

2. Any events Which do not occur jointly are the sum of all 
probabilities for individual subevents. 

3. The value of outputs (often in the form of incommensurable 
products or output) must be quantified— a difficult but 
nonetheless not impossible task. Millions of dollars in 
resources are made available for public sector expenditures 
in order that some given task with an alleged value might 
be performed. ..The very commitment of economic resources 
tacitly provides im^ )ted shadow values. 

There are rather significant differences between statistical 
|)robabilities and judgmental uncertainties. The former is best 
termed an objectivistic view of statistical decision making, 
whereas the latter is. a subjectivistic interpretation of probability. 
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Many scholars with v.ast experience in management science and in 
other related disciplines concentrate only upon objective, statistical 
probability analysis— a comfortable, logical, and empirically *'firm" 
view of the world. Unfortunately, the objectivistic view also crucial^ 
relationships as givens, including the interaction between objectivistic 
and subjectivistic data as well as the singular influence of the latter 
type of judgmental uncertainties. Objective probabilities do 
indeed rapresent the "best of all possible worlds" for the academic 
analyst, but as models or generalizations of reality they do not ^ 
necessarily represent even "second best" solutions. Experiments 
cannot always (or even most of the time) be repeated independently 
over and over again und(^r uniform conditions. As uncomfortable as it 
may be, the real world environment is uncertain and dynamic, 
and one often is forced to view objectivistic efforts as being more 
or less useless as predictive, explanatory , management decision models 
of the optimal ity variety. 

Three of the evaluation team members and three of the project 
staff made several judgments of the impact of the project using a 
Bayesian methodology of the pi^bability model. A complete description 
of the approach will be presented in a technical report under pre- 
paration. The Tes4jlts are pfes^ted' in Tables 7, 8, 9, and 10. 
Table 7 contains the averages^ the probability that a certain degree 
of improved coping behavior was attained in the target and total 
population. Table 8 shows the average probability that each proportion 
of persons in the target and total population was materially aided 
from RFD. Table 9 contains the average probability what improved 
coping skill would be worth to the participant. Table 10 shows the 
judged and desired weight given to six goals of the project, and the 
judged degree of success with the overall goal. It will be 
recognized that the overall goal was the first objective of the 
project. The six goals were rewordings based on the other nine 
objectives of the project. \ Figure 2 presents a comparison of the 
diff'^re ice between the evaluation team and the project leaders in 
the-*r judgment of the degree of success of attainment of the overall 
goal. The following material is based on these tables and the budget 
data. \^ 

\ 

Aggregate Cost-Effectiveness df RFD. 

In what follows, we shall first summarize aggregate cost-effectiveness 
and the comment upon \ie three najor media approaches. 

Let us assume that: 

1. Costs and returns are a linear and direct function of 
persons subjected to RFD efforts. 

\ 
\ 
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2. All costs of types I and II are fixed, having a straight- 
line depreciable life of five years, with a remaining salvage 
value of 10 percent of total fixed charges (10% x $570,000). 
The $570,000 figure includes type I and II costs plus all 
overhead costs. 

3. FC = fixed charges less salvage value (= $57,000) (Cf = FC/Q 
is average fixed cfjiarge allocation where Q = the number of 



families). . 
4; VC = variable charges (Cv = VC/Q is average variable charge 

allocation). _ _ 

5. Bv = effective present value 'of returns or Q • V, where V 
represents the average annual value of RFD to a participa- 
tory family. 

6. IF ^ the interest factor. All monetary values are always 
in terms of present worth, discounted at thg social rate 
of discount (assumed here to be 6 percent). 

Under these circumstances, fixed charges assignable yearly, exclud 
ing university overhead, approximate $102,000, whereas variab-le.cost - 
charges total $138,000. Assuming tliat the number of families benefit- 
ing from RFD equals the number in the target group (50 families), 
and that an equivalent annual benefit of $400 per year per family for 
20 yea^s is applicable, the data from Table 6 reveal that: 



It is much more likely, how.ever, that some number in excess of 
1.6 percent of 3,000 exposed families benefited fully from RFD. As 
Figure 2 reveals, for each such 10 family units above the break-even 
level (Bv/C = 52 x 4,600,/$240,000 * 1 .0) of 52 family beneficiaries 
net benefits above annual costs rise by $46,000, or the ratio Bv/C 
increases by approximately .2 for each 10 unit increase in family 
beneficiaries (e.g., at 100 families; Bv/C = 1.9, at 150 families, 
Bv/C = 2.9; and at 200, Bv/C = 3.9). 



the present va^ue of benefits, Bv, for a lump sum of $^ in 
the future is A x the reciprocal of the compound interest factor \ 
(1 + i)". The present value of a uniform annual series of $V 
per year is Bv = $V x IF, where 



C" " $102,000 +$I3«,UUU 



Bv _ $230,000 



= .96 



IF = 




= 11 .47 at 6% for 20 years. 




Number of Family Units 
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Thus far, little has been said about the measurable effective- 
ness of RFD— and for good reason. First, this effort represents 
the classic case of intangible and incommensurable benefits; i.e., the 
assignment of economic value to achieving greater coping behavior is at 
best a very hazardous undertaking. The hypothetical "implied" valuation 
options discussed in the previous section suggest that efforts such 
as RFD probably are economic in nature, and perhaps substantially 
so. To sharpen our understanding of the cost-effectiveness of RFD, 
some consideration can be given to certain "subjectivistic" probability 
data gathered in an attempt to gauge project success. 

table 7 indicates, for the target population of 46 disadvantaged 
families and the other exposed population, the "most probable" 
levels of improvement in coping behavior might be in the order of 20-25 
percent for the target group and 10-15 percent for the other exposed 
population. Unfortunately, subjectively determined probabilities such 
as these are not really confirmed by the evaluation data. 

Table 7 

RFD's Impact on Improved Coping Behavior 



Degree of Coping 
Behavior Improve- 
ment 

At Least 



Probability That Stated Level of Improvement 
Was Achieved 



Target 



Others 



1/25^ 


.03 ( 


.07)* 


.28 < 


.18)* 


3% 


.15 ( 


.•^^! 


.10 ( 


.12) 


10% 


21 ( 


:.2o) 


.19 ( 


.27) 


20% 


.19 ( 


:.i2) 


.15 ( 


[.20) 


30% 


.12 ( 


:.i3) 


.08 ( 


[.07) 


40% 


.08 


[.ID 


.06 ( 


[.05) 


5Q% 


.06 


[.04) 


.04 


[.03) 


70% 


05 


[.04) 


.04 


[.03) 


80% 


.04 


(.05) 


.04 


[.05) 


80%+ 


.07 


(.03) 


.05 


[.00) 



*Data estimate of the evaluation team only, whereas data not in ( ) 
represent the weighte3 average of key project leaders and the eval- 
uation team members. 



Table 8 depicts the best "quesstimates" of project leaders 
and the evaluation team concerning the proportion of persons materially 
aided. While again subject to numerous criticisms concerning data 
reliability, it is clear that those close to the project subjectively 
judge that something between 15-25 percent of the target group and 5-15 
percent of the other exposed population received such benefits in some 
measure. 
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Table 8 

Proportion of Exposed Population Benefiting From RFD 

Proportion of Persons Probability that Stated Percent of Exposed 

Exposed to RFD Population Were Materially Aided 

"Materially Aided" Target Others 

5% .17 (.30)* .36 (.62)* 

15% .10 (.20) .14 (.16) 

25% .13 (.18) .14 (.14) 

40% .20 (.23) .17 (.04) 

60% .20 (.03) .10 (.04) 

80% .15 (.03) .08 (.00) 

80%+ .05 (.03) .01 (.00) 



*Data estimates of the evaluation team only, whereas data not in ( ) 
represent the weighted averaqe of key project leaders and the evalu- 
ation team. 

On the basis of such information and other subjective impressions 
as well as objective information, our rough conclusions are: 

1. The marginal impact of the home visit program was such as 
to double the effeotiver\ess of RFD (in terms of both 
improved coping behavior and proportion of population 
materially aided) . 

2. Of some 3,000 persons in the "other population" exposed to 

RFD, something upwards to the equivalent of 50 families (ranging 
to as many as 150 or more) benefited fully from RFD. 

Table 9 represents an attempt to quantify, in annual dollar returns, 
the effective value of improved coping behavior. On a weighted 
average basis, per unit target group benefits appear to range from 
$350-$450 annually, whereas the subjective value placed upon the 
per unit value of improved coping behavior for the other exposed popu- 
lation approximates $300-$380. 

Finally, Table 10 reveals the degree to which project team 
"members and the evaluation team weighted the goals and overall per- 
formance of RFD. Goal I ("create a viable TV-based multi-media 
reusable program") and Goal II (involve a large number of under- 
educated adults ...) appear to have received most emphasis. In 
general terms, a significant disparity in the judgment of overall goal 
achievement of "demonstrating the effectiveness of the integrated 
program" exists, where the evaluation team tends to rate level of 
achievement at about 40 percent compared to something like a 70 percent 
rating on the part of project staff. 
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Table 9 

Value Alternatives for Improving Coping Behavior 



Appraised Subjectivistic $ Probability of Stated $ Levels 

Value of Improved Coping Skill Target Other 

$ 0-100 .17* .12* 

$101 - 200 .20 .25 

< $201 - 300 .12 .18 

$301 - 500 .19 .19 

$501 - 700 .10 .13 

$701 - 900 .11 .11 

$900+ .11 -02 



*Data estimates of the evaluation team and the key project leaders. 
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Table 10 



Overall Project Performance 



(a) Goal Importance 



Evaluation Team 

Desired 



Project Leaders 



Six Alternative 
Goals 



Rank Weight Weight 



Rank 



Weight 



Goal I 

Goal II 

Goal III 

Goal IV 

Goal V 

Goal VI 



1 .37- (.18) 

2 .20 (.13) 

4 .12 (.08) 
6 .08 (.10) 

3 .13 (.13) 

5 .10 (.17) 



2 
5 
6 
3 
4 



.30 
.28 
.08 
.07 
.17 
.10 



Goal I - Create a viable television based multi-media-program 

usable in similar situations in other parts of the country. 

Goal II - Involve large numbers of adults not now able or willing 
to participate in ABE programs. 

Goal III - Assist in the development of skills that can lead 
to new careers for home study aides and other staff 

members. j. .j 

Goal IV ^ Demonstrate involvement of disadvantaged individuals 

in" the development and implementation of such a program. 
Goal V - Develop participant skills in the basic fields of 
communication and computation while improving the 
capability _of the target audience to exercise citizen- 
ship responsibilities. 
Goal VI - Develop participant skills from present proficiency 

toward eighth and twelfth grade equivalency achievement 
levels. 

(b) Overall Achievement 

Degree of Success in Achieving Goal 
of "Demonstrating effectiveness of 
integrated television, home study, 

home contact, and visit program for Probability that Stated 
rural ABE students. Level Was Achieved 



10% 

30 

50 

70 

90 



.11 
.12 
.24 
.31 
.22 
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Some C omments on Individual Media Effectiveness 

No evaluation of the home visit program can ignore the fact that 
It involved incremental costs of $112,000 (or more than $2,400 
per 46 final participants), some $98,000-or $2,100 per unit- of which 
represents variable expenditures. There is no way in which even break- 
even cost effectiveness can be obtained unless variable expenditures 
per unit can be covered. For the home visit program, potential 
Bv = $4,600, Cv = $2,100, FC = $13,800, and Q = 46. 

For the home visit program to be minimally effective at the margin, 
it would require that the incremental gain in probable benefits {P) 
increase to .46. That is, as a result of the home visit program alone, 
it is necessary that the RFD project success ratio rise to almost 
1 out of every 2 exposed persons. 

p • Bv > Cv 
p . 4,600 = $2,100 
P = .46 

It does not appear reasonable to presume gains such as increased 
success ratios up to almost 1 out of 2 families were forthcoming 
at the. margin from home visits; hence, it is difficult indeed to 
iustify the home visit portion of Project RFD. However, since one 
wnnot separate the "joint" product resultant from some combination 
of TV, Materials, and Home Visits, about all that can be said is 
nhat the burden of proof for Home Visits rests with that program and 
its rather high unit costs relative to a nominal quantity base. 

It should be kept in mind the Home Visit component was judged to 
have had quite an effect on- project impact with those it reached 
Also the costs of this component might be reduced considerably by 
more efficient use of home visitors or by^uch things as volunteer home 
visitors. 

Again considered separately, the cost-effectiveness of TV as 
a media may represent a more economic operation. Assuming some 20,000 
famil es were exposed to and benefited from no less than one program, 
t woJld reSSire only $1 per year per family in benefits spread over 
r20-year benefit lifetime to achieve a benefit/cost ratio equal to 
1.3. 

Bv $11.47 _ 1 , 
C~rQ $ 8.60 " 
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Alternatively, if only 1 out of 10 exposed families acquired annual 
benefits equal to $20 per year and such families accure the 
average costs of 10 exposed family units, each $1 in costs would 
generate $2.67 in benefits. 



A tempering ispect to this favorable ratio is that the assumption 
of a five year life for the materials is least tenable for television. 

Similar conclusions apply to the material media, which include 
action line as well as a large variety of written materials. Assuming 
that simply 3,000 pieces of material were distributed at a total 
cost of $256,000 or an average cost of $85, break-even cost-effectiveness 
would require benefits of approximately $7.40 per year for each 
material recipient, or some $15.00 annually if only one-half of 
all recipients actually benefited (or if a minimum Bv/C = 2 was 
stipulated and all recipients benefited. 

In summary, RFD appears to have been a reasonably cost-effective 
effort— in no small measure because (a) demand for such services is 
substantial and (b) a relatively large number of persons were exposed 
to RFD (excluding the Home Visit program). These conclusions depend, 
however, upon the quantitative magnitudes used, and while much about 
the data is admittedly difficult to pinpoint with accuracy and may even 
be ill-defined, RFD does appear to have been a worthy research and 
demonstration effort. This is not to say, however, that the greatest 
possible efficiences were achieved. Much about the Home Visit program 
was excessively costly relative to the potential benefits that could 
have been generated; the reusable output from TV as a media may be 
severely limited and if so, costs would have to be depreciated over, 
'less than 5 years; and the topical and qualitative features of much of 
the written materials could be improved upon. ' 



Bv _ $229.00 



= 2.67 



C f lOQ $ 86.00 
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Recommendations (, 

Our study of this project has -uggested several recommendations of 
varied specificity. Most of the recommendations are of feared as points 
worthy of consideration by people working in this area. 

1. The staff of the Project RFD should be encouraged and supported 
in continuing their work in this area. They have now developed 
an expertise that should not be lost from Uck of use. 

2. Television and home study materials seem to have favorable 
cost/effectiveness possibilities, but it is more difficult 
to justify home visitors on a cost/effectiveness basis. 

3. We would recommend that an educational type project should 

be identified with and administered in a unit that has expertise 
in the content and audience rather than a materials production- 
unit. 

4. Obviously, all components of a project should be ready when 
needed. The experience of RFD points to the need for planning 
sufficient lead time to insure that all components are ready. 

\ » 

5. Educational programs should be planned with the target ^ 
audience not for them. 

6. Evaluation activities should be closely tied to the on-going 
administration of the project so that they are kept relevant. ^ 
Furthermore, the evaluation should be planned so that it is 
largely done in time for use in the disseminatron activities. 

7. Testing and other kinds of data gathering with adults are 
problems. Unobtrusive measures should be used if possible. 

. Where direct data gathering is to be done, the project should 
be straightforward and honest in its approach. It should 
not do things it said it wouldn't do. . 

8. Educational activities are prodtret-s- to be'marketed, and 
sophisticated, approaches to marketing! them seem to be justified. 

9. The -integration of various edlicatiorHil media is'easier said 
than done. Experimentation is still neede.d in this'* area, 
especially in difficult to control situations like home study. 
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10, When using a home visitor approach it would seem that two 
things might be considered to optimize effects, 

(a) ' Select for the treatment those who are quite aware 

of what is involved, and who have indicated an interest 
in being involved based on their awareness, or 

(b) Have a warm-up period»in which the home visitor is able 
to establish the needed rappert before the rest of 

the treatment is started, ^ 

n. There is evidence to recommend that techniques like enrollment 
, ' and stated commitment are conducive ^o participation and per- 
sistence by adults in' educational programs. 
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LiVt of Recommended Project Materials for In-Depth Information. - 
Limited copies are available from the project.' Items 1, 2, 3, and 
6 will likely be in ERIC. 

- 1. RFD - The First Year - Provides descriptions of the planning 
anH" theTonceptualizaticn of the project. , j» 

' 2\ RFD - Rural Family Developiyent - Second Year Report, June, 1971. 
■ jrgood~aesc'ription of the operation of the project.. Several- ^ 
of the reports of theTinternal evaluators are included. 

3. The RFD System - This report provides a qood overview of 
each component. Cost estimates are inc. tided. 

4. Newsletters - A complete set of the newsletters published monthly 
- will provide a good overview of the project. 

5. The Almanac - A copy of the almanac is useful reading in that .„ 
it gives an idea of the content emphases of the project'. 

"\, 

6. *" \ third year report will be written. It will include 

additional reports of the internal evaluators. 



